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ABSTRACT 

We study the randomized version of a computation model (intro- 
duced in (91 1101 ) that restricts random access to external memory 
and internal memory space. Essentially, this model can be viewed 
as a powerful version of a data stream model that puts no cost 
on sequential scans of external memory (as other models for data 
streams) and, in addition, (like other external memory models, but 
unlike streaming models), admits several large external memory 
devices that can be read and written to in parallel. 

We obtain tight lower bounds for the decision problems set equal- 
ity, multiset equality, and checksort. More precisely, we show that 
any randomized one-sided-error bounded Monte Carlo algorithm 
for these problems must perform fl(logA') random accesses to ex- 
ternal memory devices, provided that the internal memory size is at 
most 0(\/~N /XogN), where N denotes the size of the input data. 

From the lower bound on the set equality problem we can infer 
lower bounds on the worst case data complexity of query evalu- 
ation for the languages XQuery, XPath, and relational algebra on 
streaming data. More precisely, we show that there exist queries 
in XQuery, XPath, and relational algebra, such that any (random- 
ized) Las Vegas algorithm that evaluates these queries must per- 
form fl(logA') random accesses to external memory devices, pro- 
vided that the internal memory size is at most 0(\/~N /log A). 

Categories and Subject Descriptors 

F.1.3 [Computation by Abstract Devices]: Complexity Measures 
and Classes; F.l.l [Computation by Abstract Devices]: Models 
of Computation 

General Terms 

Theory, Languages 

Keywords 

complexity, data streams / real-time data, query processing / query 
optimization, semi-structured data, XML 



1. INTRODUCTION 

Today's hardware technology provides a hierarchy of storage 
media from tapes and disks at the bottom through main memory 
and (even on-CPU) memory caches at the top. Storage media from 
different levels of this memory hierarchy considerably differ in 
price, storage size, and access time. Currently, the most pronounced 
performance and price (and consequently also size) gap is between 
main memory and the next-lower level in the memory hierarchy, 
usually magnetic disks which have to rely on comparably slow, 
mechanical, physically moving parts. One often refers to the up- 
per layers above this gap by internal memory and the lower layers 
of the memory hierarchy by external memory. The technological 
reality is such that the time for accessing a given bit of information 
in external memory is five to six orders of magnitude larger than 
the time required to access a bit in internal memory. Apart from 
this, concerning external memory, random accesses (which involve 
moving the disk head to a particular location) are significantly more 
expensive than sequential scans. 

Modern software and database technology uses clever heuristics 
to minimize the number of accesses to external memory and to pre- 
fer streaming over random accesses to external memory. There has 
also been a wealth of research on the design of so-called external 
memory algorithms (cf., e.g. 1 16, 18, 13 1). The classes considered 
in computational complexity theory, however, usually do not take 
into account the existence of different storage media. In (9] 1101 . 
we introduced a formal model for such a scenario. The two most 
significant cost measures in our setting are the number of random 
accesses to external memory and the size of the internal memory. 
Our model is based on a standard multi-tape Turing machine. Some 
of the tapes of the machine, among them the input tape, represent 
the external memory. They are unrestricted in size, but access to 
these tapes is restricted by allowing only a certain number r(N) 
(where N denotes the input size) of reversals of the head direc- 
tions. This may be seen as a way of (a) restricting the number of 
sequential scans and (b) restricting random access to these tapes, 
because each random access can be simulated by moving the head 
to the desired position on a tape, which involves at most two head 
reversals. The remaining tapes of the Turing machine represent the 
internal memory. Access to these internal memory tapes (i.e., the 
number of head reversals) is unlimited, but their size is bounded by 
a parameter s(N). We let ST(r(N),s(N),0(l)) denote the class 
of all problems that can be solved on such an (r(N),s(N),0(l))- 
bounded Turing machine, i.e., a Turing machine with an arbitrary 
number of external memory tapes which, on inputs of size N, per- 
forms less than r(N) head reversals on the external memory tapes, 
and uses at most space s(N) on the internal memory tapes. 

The astute reader who wonders if it is realistic to assume that the 
external memory tapes can be read in both directions (which disks 
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cannot so easily) and that a sequential scan of an entire external 
memory tape accounts for only one head reversal (and thus seems 
unrealistically cheap) be reminded that this paper's main goal is 
not to design efficient external memory algorithms but, instead, to 
prove lower bounds. Thus, considering a rather powerful computa- 
tion model makes our lower bound results only stronger. 

In the present paper, we prove lower bounds for randomized 
computations (i.e., computations where in each step a coin may 
be tossed to determine the next configuration) in a scenario with 
several storage media. To this end, we introduce the complexity 
class RST(r(N),s(N),0(l)), which consists of all decision prob- 
lems that can be solved by an (r(N) ,s(N) ,0(1)) -bounded random- 
ized Turing machine with one-sided bounded error, where no false 
positive answers are allowed and the probability of false negative 
answers is at most 0.5 (in the literature, such randomized algo- 
rithms are often called one-sided-error Monte Carlo algorithms, 
cf. 1121 ). To also deal with computation problems where an out- 
put (other than just a yes/no answer) has to be generated, we write 
LasVegas-RST(r(N),s(N),0(l)) to denote the class of all func- 
tions / for which there exists an (r(N), s(N), 0(l))-bounded ran- 
domized Turing machine that, for every input word w, (a) always 
produces either the correct output f(w) on one of its external mem- 
ory tapes or gives the answer "I don't know" and (b) gives the 
answer "I don't know" with probability at most 0.5 (in the litera- 
ture, such randomized algorithms are sometimes called Las Vegas 
algorithms, cf. 1121 ). 

Contributions: Our first main result is a lower bound for three 
natural decision problems: The set equality problem and the mul- 
tiset equality problem ask whether two given (multi)sets of strings 
are equal, and the checksort problem asks, given two sequences of 
strings, whether the second is a sorted version of the first. 
We show (Theorem[6]l that neither problem is contained in 

RST(o(l0gAf),0( 1 ^),O(l)). This lower bound turns out to be 
tight in the following senses: 

• If the number of sequential scans (i.e., head reversals) increases 
from o(logA') to O(logA'), then each of the three problems can 
be solved with only constant internal memory and without using 
randomization. In other words (see Corollary [7), the (multi)set 
equality problem and the checksort problem belong to 
ST(0(logjV),O(l),0(l)). 

• When using randomization with the complementary one-sided 
error model, i.e., machines where no false negative answers are 
allowed and the probability of false positive answers is at most 
0.5, then the multiset equality problem can be solved with just 
two sequential scans of the input (and without ever writing to ex- 
ternal memory), and internal memory of size O(logA'). In other 
words (Theorem[8ta)), the multiset equality problem belongs to 
co-RST(2,0(logAO,l). 

• When using nondeterministic machines, then (multi)set equality 
and checksort can be solved with three sequential scans on two 
external memory tapes and internal memory of size O(logA'). In 
other words (Theorem[8jb)), the (multi)set equality problem and 
the checksort problem belong to NST(3,0(logA'),2). 

As a consequence, we obtain a separation between the determinis- 
tic, the randomized, and the nondeterministic ST(- • ■ ) classes (Cor- 
ollary |9). 

Our lower bound for the checksort problem, in particular, im- 
plies that the sorting problem (i.e., the problem of sorting a se- 
quence of input strings) does not belong to the complexity class 



LasVegas-RST(o(logN),0{-^),0(l)) and thus generalizes the 
main result of 1101 to randomized computations. 

Our lower bound for the set equality problem leads to the fol- 
lowing lower bounds on the worst case data complexity of database 
query evaluation problems in a streaming context: 

• There is an XQuery query Q such that the problem of evaluating 
Q on an input XML document stream of length N does not be- 
long to the class LasVegas-RST(o(logN), O(M^), 0(1)) (The- 
orem [72] >. 

Speaking informally, this means that, no matter how many external 
memory devices (of arbitrarily large size) are available, as long as 

the internal memory is of size at most O(-^jy), every randomized 
algorithm that produces the correct query result with probability at 
least 0.5 will perform Q.(logN) random accesses to external mem- 
ory. We obtain analogous results for relational algebra queries and 
for the node-selecting XML query language XPath: 

• There is a relational algebra query Q such that the problem 
of evaluating Q on a stream consisting of the tuples of the in- 
put database relations does not belong to the complexity class 

LasVegas-RST(o(logN),0{^j),0{l)), where N denotes the 
total size of the input database relations. Furthermore, this bound 
is tight with respect to the number of random accesses to exter- 
nal memory, as the data complexity of every relational algebra 
query belongs to ST(0(logA r ), 0(1), 0(1)) (TheoremfTTTl. 

• There is an XPath query Q such that the problem of filtering an 
input XML document stream with Q (i.e., checking whether at 
least one node of the document matches the query) does not be- 
long to the class co-RST(o(logAT),0( I ^) > 0(l)) (Theo- 
rem |T3j. 

This means that there is an XPath query Q such that, no matter how 
many external memory devices (of arbitrarily large size) are avail- 

able, as long as the internal memory is of size at most O(^jj), 
every randomized algorithm which accepts every input document 
that matches Q, and which rejects documents not matching Q with 
probability > 0.5, will perform fl(logA') random accesses to exter- 
nal memory. 

Related Work: Obviously, our model is related to the bounded re- 
versal Turing machines, which have been studied in classical com- 
plexity theory (see, e.g., 1191 17)). However, in bounded reversal 
Turing machines, the number of head reversals is limited on all 
tapes, whereas in our model there is no such restriction on the in- 
ternal memory tapes. This makes our model considerably stronger, 
considering that in our lower bound results we allow internal mem- 
ory size that is polynomially related to the input size. Furthermore, 
to our best knowledge, all lower bound proofs previously known 
for reversal complexity classes on multi-tape Turing machines go 
back to the space hierarchy theorem (cf., e.g., 1 1 71 17|) and thus 
rely on diagonalization arguments, and apply only to classes with 
Co(logN) head reversals. In particular, these lower bounds do not 
include the checksort problem and the (multi)set equality problem, 
as these problems can be solved with O(logA') head reversals. 

In the classical parallel disk model for external memory algo- 
rithms (see, e.g., 1 1 8 . 13, 16|), the cost measure is simply the num- 
ber of bits read from external memory divided by the page size. 
Several refinements of this model have been proposed to include 
a distinction between random access and sequential scans of the 
external memory, among them Arge and Bro Miltersen's external 
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memory Turing machines |3|. We note that their notion of external 
memory Turing machines significantly differs from ours, as their 
machines only have a single external memory tape and process in- 
puts that consist of a constant number m of input strings. Strong 
lower bound results (in particular, for different versions of the sort- 
ing problem) are known for the parallel disk model (see 1 18 1 for an 
overview) as well as for Arge and Bro Miltersen's external mem- 
ory Turing machines (3). However, to the best of our knowledge, 
all these lower bound proofs heavily rely on the assumption that 
the input data items (e.g., the strings that are to be sorted) are indi- 
visible and that at any point in time, the external memory consists, 
in some sense, of a permutation of the input items. We emphasize 
that the present paper's lower bound proofs do not rely on such an 
indivisibility assumption. 

Strong lower bounds for a number of problems are known in the 
context of data streams and for models which permit a small num- 
ber of sequential scans of the input data, but no auxiliary external 
memory (that is, the version of our model with no extra external 
memory tapes apart from the input tape) fT31 |2l [TT1 |4l [Tfjl [51 151 [T1 
[£)■ All these lower bounds are obtained by communication com- 
plexity. Note that in the presence of at least two external memory 
tapes, communication between remote parts of memory is possi- 
ble by simply copying data from one tape to another and then re- 
reading both tapes in parallel. These communication abilities of 
our model spoil any attempt to prove lower bounds via communi- 
cation complexity, which is the tool of choice both for computation 
models permitting few scans but no auxiliary external memory, and 
for 1-tape Turing machines. 

The deterministic ST(- • • )-classes were introduced in 1101 l9l- In 
1 9 1 we studied those classes where only a single external memory 
tape is available and used methods from communication complex- 
ity to obtain lower bounds for these classes. The main result of 
1101 was a lower bound for the sorting problem concerning the de- 
terministic ST(- • • )-classes with an arbitrary number of external 
memory tapes. An important tool for proving this bound was to 
introduce deterministic list machines as an intermediate machine 
model. An overview of the methods used and the results obtained 
in t9l 1101 was given in [8]. The present paper builds on 1101 . as 
it considers ST(- • • )-classes with an arbitrary number of external 
memory tapes and it uses list machines as a key tool for proving 
lower bound results. However, the results presented here go sig- 
nificantly beyond those obtained in 1101 . Here we obtain lower 
bounds for decision problems in the randomized versions of the 
model. The main result of 1101 is that the sorting problem does not 

belong to ST(o(logA0,O(J^),O(l)), and the proof given there 
heavily relies on the fact that the machines are deterministic and 
the output of the sorting problem cannot be generated within the 
given resource bounds. In contrast to the present paper's approach, 
the proof method of 1101 neither works for decision problems, i.e. 
problems where no output is generated, nor for randomized com- 
putations. Finally, let us remark that the main result of 1 10 1 can 
be obtained as an immediate corollary of the present paper's lower 
bound for the checksort problem. 

Organization: After introducing the deterministic, the nondeter- 
ministic, and the randomized ST(- • • ) classes in Section|2] we for- 
mally state our main lower bounds for decision problems in Sec- 
tion^ In Section|4]we use these results to derive lower bounds on 
the data complexity of query evaluation for the languages XQuery, 
XPath, and relational algebra. The subsequent sections are devoted 
to the proof of the lower bound on the decision problems (multi)set 
equality and checksort: In Section|5J|6] and|7]we introduce random- 
ized list machines, show that randomized Turing machines can be 



simulated by randomized list machines, and prove that randomized 
list machines can neither solve the (multi)set equality problem nor 
the checksort problem. Afterwards, in Section [8] we transfer these 
results from list machines to Turing machines. We close with a few 
concluding remarks and open problems in Section|9] 

The present paper is the full version of the extended abstract pub- 
lished in the proceedings of the 25th ACM Sigact-Sigart Sympo- 
sium on Principles of Database Systems (PODS'06). 

2. COMPLEXITY CLASSES 

We write N to denote the set of natural numbers (that is, nonneg- 
ative integers). 

As our basic model of computation, we use standard multi-tape 
nondeterministic Turing machines (NTMs, for short); cf, e.g., 1171 . 
The Turing machines we consider will have t + u tapes. We call the 
first t tapes external memory tapes (and think of them as represent- 
ing / disks). We call the other ;/ tapes internal memory tapes. The 
first tape is always viewed as the input tape. 

Without loss of generality we assume that our Turing machines 
are normalized in such a way that in each step at most one of its 
heads moves to the left or to the right. 

Let T be an NTM and p a finite run of T, Let i > 1 be the number 
of a tape. We use rev(p,r) to denote the number of times the head 
on tape i changes its direction in the run p. Furthermore, we let 
space (p, i) be the number of cells of tape i that are used by p. 

Definition 1 ((r,s,t) -bounded TM). Let r,s : N -> N and t € N. A 

(nondeterministic) Turing machine T is (r,s,t)-bounded, if every 
run p of T on an input of length N (for arbitrary N £ N) satisfies the 
following conditions: (1) p is finite, (2) 1 +£/=i rev(p, /) < r(N), 
ant{3(3) Eii"j-i space(p, i) < s(N), where t + u is the total number 
of tapes of T. H 

Definition 2 (ST( ) and NST( ) classes). 

Let r, s : N — » N and t e N. A decision problem belongs to the class 
ST(r,s,t) (resp., NST(r, s,t)), if it can be decided by a deterministic 
(resp., nondeterministic) (r,s,t) -bounded Turing machine. H 

Note that we put no restriction on the running time or the space 
used on the first t tapes of an (r, s,/)-bounded Turing machine. The 
following lemma shows that these parameters cannot get too large. 

Lemma 3 (| 10 1). Let r, s : N -> N and 1 6 N, and let T be an (r,s,t)- 
bounded NTM. Then for every run p = (pi , . . . , pi) ofT on an input 
of size N we have £<N- 2°W-( t+s W)) and thus £? =1 space(p, i) 

In (Tp |, the lemma has only been stated and proved for deter- 
ministic Turing machines, but it is obvious that the same proof also 
applies to nondeterministic machines (to see this, note that, by def- 
inition, every run of an (r,s,t) -bounded Turing machine is finite). 

In analogy to the definition of randomized complexity classes 
such as the class RP of randomized polynomial time (cf, e.g., 
1171 ), we consider the randomized versions RST(- • • ) and LasVe- 
gas-RST{- ■ ■ ) of the ST(- • • ) and NST(- • • ) classes. The following 
definition of randomized Turing machines formalizes the intuition 
that in each step, a coin can be tossed to determine which particular 
successor configuration is chosen in this step. For a configuration 
y of an NTM T, we write Next 7- (y) to denote the set of all con- 
figurations Y that can be reached from y in a single step. Each 

It is convenient for technical reasons to add 1 to the number 
£j =1 rev(p,;) of changes of the head direction here. As defined 
here, r(N) thus bounds the number of sequential scans of the ex- 
ternal memory tapes rather than the number of changes of head 
directions. 
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such configuration / £ Next7-(y) is chosen with uniform probabil- 
ity, i.e., Pr(y^ r /) = l/|Next r (y)|. For a run p = (p u ..,p e ), 
the probability Pr(p) that T performs run p is the product of the 
probabilities Pr(p,- —+t P/+1 )> f° r all i < I. For an input word w, 
the probability that T accepts w (resp., that T outputs w') is defined 
as the sum of Pr(p) for all accepting runs p of T on input w (resp., 
of all runs of T on w that output w'). We say that a decision prob- 
lem L is solved by a ( \ , 0)-RTM if, and only if, there is an NTM T 
such that every run of T has finite length, and the following is true 
for all input instances w: If w £ L, then Pr(T accepts w) > 1 /2; if 
w ^ L, then Pr(T accepts w) = 0. Similarly, we say that a function 
/ : Z* — » Z* is computed by a LajVfcgas-RTM if, and only if, there 
is an NTM T such that every run of T on every input instance w 
has finite length and outputs either f(w) or "I don't know", and 
Pr(r outputs f(w)) > 1/2. 

Definition 4 (RST( ) and La* Veg«s-RST( )). 

Let r, s : N — > N and f £ N. 

(a) A decision problem L belongs to the class RST(r, s, t), if it can 
be solved by a (|,0)-RTM that is (r,s,t) -bounded. 

(b) A function / : Z* — > Z* belongs to LasVegas-RST(r,s,t), if it 
can be solved by a Las Vegas-KTM that is (r, s, ?)-bounded. H 

As a straightforward observation one obtains: 
Proposition 5. For all r, s : N — » N and f £ N, 
ST(r,s,f) C RST(r,j,f) C NST(r,s,t). H 
For classes R and 5 of functions we let ST(R,S,t) := Urefl ses 
ST{r,s,t) and ST(if, 5,0(1)) :=UreN ST(«,S,f). Analogous nota- 
tions are used for the NST(- • • ), RST(- • • ), and LasVegas-RST(- ■ ■ ) 
classes, too. 

As usual, for every (complexity) class C of decision problems, 
co-C denotes the class of all decision problems whose complements 
belong to C. Note that the RST(- • • )-classes consist of decision 
problems that can be solved by randomized algorithms that allow a 
moderate number of false negatives, but no false positives. In con- 
trast to this, the co-RST(- • ■ )-classes consist of problems that can 
be solved by randomized algorithms that allow a moderate number 
of false positives, but no false negatives. 

From Lemma [3] one immediately obtains for all functions r,s 
with r(N) ■ s(N) £ O(logiV) that ST(r,s,0(l)) C PTIME, 
RST(r,s,0(l)) C RP, and NST(r,s,0(l)) C NP (where PTIME, 
RP, and NP denote the class of problems solvable in polynomial 
time on deterministic, randomized, and nondeterministic Turing 
machines, respectively). 

3. LOWER BOUNDS FOR 
DECISION PROBLEMS 

Our first main result is a lower bound for the (multi)set equality 
problem as well as for the checksort problem. The (multi)set equal- 
ity problem asks if two given (multi)sets of strings are the same. 
The checksort problem asks for two input lists of strings whether 
the second list is the lexicographically sorted version of the first 
list. We encode inputs as strings over the alphabet {0, 1,#}. For- 
mally, the ( multiset equality and the checksort problem are defined 
as follows: The input instances of each of the three problems are 

Instance: v\# - ■ ■ #v m #Vj # • • • #v' m #, 

where m>0, and v,-, vj £ {0, 1}* (for all i < m) 

and the task is to decide the following: 

Set-Equality problem: 

Decide if {v\ , . . . , v m } = {v\ v' m }. 



Multiset-Equality problem: 

Decide if the multisets {vi,...,v m } and {v'j , . . . , v m } are 
equal (i.e., they contain the same elements with the same 
multiplicities). 

Check-Sort problem: 

Decide if Vj , . . . , v' m is the lexicographically sorted (in as- 
cending order) version of vi, . . . , v m . 

For an instance Vi#---v m #Vj#---vJ„# of the above problems, we 
usually let N = 2m + KLi (|v,-| + |v;|) denote the size of the input. 
Furthermore, in our proofs we will only consider instances where 
all the Vj and vj have the same length n, so that N = 2m ■ (n + 1). 

The present paper's technically most involved result is the fol- 
lowing lower bound: 

Theorem 6. Let r,s : N — > N such that r(N) £ o(logiV) and s(N) £ 
o[s/~N /r(N)). Then, none of the problems CHECK-SORT, 
Set-Equality, Multiset-Equality belongs to the class 
RST(r(JV),s(A0,O(l)). H 

Sections [5}{8] are devoted to the proof of Theorem [6] The proof 
uses an intermediate computation model called list machines and 
proceeds by (1) showing that randomized Turing machine compu- 
tations can be simulated by randomized list machines that have 
the same acceptance probabilities as the given Turing machines 
and (2) proving a lower bound for (MULTI)SET-EQUALITY and 
CHECK-SORT on randomized list machines. 

By applying the reduction used in 1101 Theorem 9], we obtain 
that the lower bound of Theorem [6] also applies for the "SHORT" 
versions of (MULTI)SET-EQUALITY and CHECK-SORT, i.e., the 
restrictions of these problems to inputs of the form v\#---v m # 
v'j#- • • v' m #, where each v; and vj is a 0-1-string of length at most 
c- logm, and c is an arbitrary constant > 2. By using the stan- 
dard merge sort algorithm, one easily obtains that the "SHORT" 
versions of (MULTI)SET-EQUALITY and CHECK-SORT belong to 
ST(0(logAT),0(logiV),3). Moreover, in Lemma 7] it has been 
shown that the (general) sorting problem can be solved by an 
(0(logA^),0(l),2)-bounded deterministic Turing machine. As an 
immediate consequence, we obtain: 

Corollary 7. Set-Equality, Multiset-Equality, Check- 
Sort, and their "SHORT" versions, are in ST(0(logA'),0(l),2), 
butnotinRST(o(logN),0(^N/logN),0(l)). H 
A detailed proof can be found in Appendix lEl 
As a further result, we show that 

Theorem 8. (a) Multiset-Equality belongs to 
co-RST(2,O(logA0,l) C co-NST(2,0(logAT),l). 

(b) Each of the problems MULTISET-EQUALITY, Check-Sort, 
Set-Equality belongs to NST(3,0(logJV),2). H 

Proof: We apply fairly standard fingerprinting techniques and 
show how to implement them on a (2,0(\ogN), l)-bounded ran- 
domized Turing machine. Consider an instance v\#. . .#v m #v'j#. . .# 
vj„# of the MULTISET-EQUALITY problem. For simplicity, let us 
assume that all the Vj and Vj have the same length n. Thus the in- 
put size N is 2 ■ m • (n + 1). We view the Vj and vj as integers in 
{0, . . . , 2" — 1} represented in binary. 

We use the following algorithm to decide whether the multisets 
{vi , . . - , v m } and {vj , . . . , V m } are equal: 

(1) During a first sequential scan of the input, determine the input 
parameters n, m, and N. 

(2) Choose a prime p\ <k:=m^-n- log(m 3 • n) uniformly at ran- 
dom. 



4 



(3) Choose an arbitrary prime pi such that 3k < P2 < 6k. Such a 
prime exists by Bertrand's postulate. 

(4) Choose x £ {1,.. . ,P2~ 1} uniformly at random. 

(5) For 1 < i < m, let e; = (v,- mod p\) and e' l = (v| mod p\). If 

m m 

£v- = XX< mod P2 (i) 

i=l i=l 
then accept, else reject. 

Let us first argue that the algorithm is correct (for sufficiently large 
m,n): Clearly, if the multisets {vi,...,V m } and {v'j , . . . , v' m } are 
equal then the algorithm accepts. On the other hand, if they are dis- 
tinct, the probability that the multisets {e\ , . . . , e m } and {e\ , . . . , e' m } 
are equal is 0(1 /m). This is due to the following. 

CLAIM 1. Letn,m£N,k = m^-n-\og(m i -n),andO<v\,..,v m , 
v'l ,.. , v' m < 2". Then for a prime p<k chosen uniformly at random, 
Pr(3i , j < m with V; ^ v'j and v,- = v'j mod p) < 0(l/m) . 

Proof: We use the following well-known result (see, for example, 
Theorem 7.5 of [0]): Letn,£ eN,t = <-n- log(f • n), and < x < 
2". Then for a prime p < k chosen uniformly at random, 

Pr(x = Omod p) < 0(\). 

The claim then follows if we apply this result with t = m} simulta- 
neously to the at most m 2 numbers x = Vj — v'j with v; ^ v'j. □ 

To proceed with the proof of Theorem Wta% . suppose that the two 
multisets are distinct. Then the polynomial 

m m 
q(X) = I-V' 

!=1 1=1 

is nonzero. Note that all coefficients and the degree of q(X) are 
at most k < p2- We view q(X) as a polynomial over the field F p2 . 
As a nonzero polynomial of degree at most p\, it has at most p\ 
zeroes. Thus the probability that q(x) = for the randomly chosen 
x G {1, . .. ,P2 — 1} is at most p\/(p2 — 1) < 1/3. Therefore, if the 
multisets {ei , . . . ,e m } and {e'j , . . . ,e' m } are distinct, the algorithm 
accepts with probability at most 1/3, and the overall acceptance 
probability is at most 

for sufficiently large m. This proves the correctness of the algo- 
rithm. 

Let us now explain how to implement the algorithm on a (2, 
O(logA'), l)-bounded randomized Turing machine. Note that the 
binary representations of the primes p\ and p2 have length O(logA'). 
The standard arithmetical operations can be carried out in linear 
space on a Turing machine. Thus with numbers of length O(logA'), 
we can carry out the necessary arithmetic on the internal memory 
tapes of our (2,0(logA'), l)-bounded Turing machine. 

To choose a random prime p\ in step (2), we simply choose a 
random number < k and then test if it is prime, which is easy in lin- 
ear space. If the number is not prime, we repeat the procedure, and 
if we do this sufficiently often, we can find a random prime with 
high probability. Steps (3) and (4) can easily be carried out in inter- 
nal memory. To compute the number e; in step (5), we proceed as 
follows: Suppose the binary representation of v,- is v ; ( n -\) ■ ■ ■ v /.0> 
where v,- o is the least significant bit. Observe that 

ra-1 

e i = (<X 2i ' v hj) modpi). 
j=0 



We can evaluate this sum sequentially by taking all terms modulo 
p\\ this way we only have to store numbers smaller than p\. This 
requires one sequential scan of v, and no head reversals. 

To evaluate the polynomial YIILi 3 ^' modulo p2, we proceed as 
follows: Let f,- = (x ej mod p\) and .s; = ((L; = i k) m °d P\)- Again 
we can compute the sum sequentially by computing e,-, f;, and 
s i = (i s i- \ m °d p\) for i = 1, ... ,m. We can evaluate YIIL\ 
analogously and then test if |QJ holds. This completes the proof of 
part ((of of Theorem [8] 

(b): Let w be an input of length N, w := vi#V2#. . .#v,„# 
v'j #v' 2 # . . . #^ m #- Note that the multisets { v\ , . . . , v m } and {v'j , ... , 
v' m } are equal if and only if there is a permutation 7t of {1, ... ,m} 
such that for all i 6 {1, • ■ • V; = v w;y The idea is to "guess" 
such a permutation n (suitably encoded as a string over {0, 1,#}), 
to write sufficiently many copies of the string u := n#w onto the 
first tape, and finally solve the problem by comparing v; and 
bitwise, where in each step we use the next copy of u. 

A (3,0(logA'),2)-bounded nondeterministic Turing machine M 
can do this as follows. In a forward scan, it nondeterministically 
writes a sequence u\ , 112, . . . , uy of £ := m + N ■ m many strings on 
its first and on its second tape, where 

ui := 7r,;i#.. .#iz ijn #v itl #. . .#v ; . m #v; ]#. . .#v- m # 

for binary numbers Ttyj from {l,...,m}, and bit strings vy and 
v'i j of length at most A'. While writing the first N ■ m strings, it 
ensures that for every i 6 {1, . . . ,N ■ m}, either v ; \i/N~\ ar, d K , 
coincide on bit ((i — 1) mod N) + 1, or that both strings have no 
such bit at all. While writing the last m strings, it ensures that for all 
(' G {l,...,m} and j e {i+ l,...,m}, 7t N . m+i j ^ 7t N . m+iJ . Finally, 
M checks in a backward scan of both external memory tapes that 
iii = for all /£ {2,... ,£}, and that vij = vj and v'j • = v'j for 
all;G{l,...,m}. 

The SET-EQUALITY problem can be solved in a similar way. 

Deciding CHECK-SORT is very similar: the machine addition- 
ally has to check that is smaller than or equal to v'j for all j G 

{('-)- 1, ... ,m}. This can be done, e.g., by writing A' • Y!!=i * ad- 
ditional copies of 11, and by comparing vj and v'j bitwise on these 
strings for each i and j G {('+ 1, ...,m}. □ 

Theorems [6] and [8] in particular, immediately lead to the follow- 
ing separations between the deterministic, randomized, and nonde- 
terministic ST(- • • ) classes: 

Corollary 9. Let r,s : N — > N with r(N) G o(logA') and s(N) G 
o(^N/r{N)) nO(logiV). Then, 

(a) RST(0{r),0{s),0{l)) ± co-RST(0(r),0(i),0(l)), 

(b) ST(0(r),0(a),0(l)) £ RST(0(r),0(*),0(l)) £ 

NST(0(r),0(*),0(l)). 

The (straightforward) proof can be found in Appendix lEl 

Let us note that the lower bound of Theorem [6] for the problem 
CHECK-SORT in particular implies the following generalization of 
the main result of 1 10 1 to randomized computations: 

Corollary 10. The sorting problem (i.e., the problem of sorting a 
sequence of input strings) does not belong to the class LasVegas- 
RST(o(logiV),0(v / ivyiogA'),0(l)). H 

The (straightforward) proof can be found in Appendix IE1 

4. LOWER BOUNDS FOR 
QUERY EVALUATION 

Our lower bound for the SET-EQUALITY problem (Theorem[6]l 
leads to the following lower bounds on the worst case data com- 



plexity of database query evaluation problems in a streaming con- 
text: 

Theorem 11 (Tight Bound for Relational Algebra). 

(a) For every relational algebra query Q, the problem of eval- 
uating Q on a stream consisting of the tuples of the input 
database relations can be solved in ST(0(logiV),0(l),0(l)). 

(b) There exists a relational algebra query Q 1 such that the prob- 
lem of evaluating Q' on a stream of the tuples of the input 
database relations does not belong to the class LasVegas- 
RST{o(logN),0{</N/logN),0(l)). H 

Proof: (a): It is straightforward to see that for every relational 
algebra query Q there exists a number cq such that Q can be evalu- 
ated within cq sequential scans and sorting steps. Every sequential 
scan accounts for a constant number of head reversals and constant 
internal memory space. Each sorting step can be accomplished us- 
ing the sorting method of (7] Lemma 7] (which is a variant of the 
merge sort algorithm) with O(logA') head reversals and constant 
internal memory space. Since the number cq of necessary sort- 
ing steps and scans is constant (i.e., only depends on the query, 
but not on the input size AO, the query Q can be evaluated by an 
(O(logiV), 0(1), 0(l))-bounded deterministic Turing machine. 

(b): Consider the relational algebra query 

Qf := (Ri-R 2 )U(R 2 -R l ) 

which computes the symmetric difference of two relations R\ and 
R 2 . Note that the query result is empty if, and only if, R\ = R 2 . 
Therefore, any algorithm that evaluates Q 1 solves, in particular, 
the SET-EQUALITY problem. Hence, if 0/ could be evaluated in 
La5Vega5-RST(o(logA'),0(v / iV/logA'),0(l)), then Set-Equal- 
ity could be solved in RST(o(logiV), 0{\fN/\ogN), 0(1)), con- 
tradicting Theorem[6] □ 

We also obtain lower bounds on the worst case data complexity 
of evaluating XQuery and XPath queries against XML document 
streams: 

Theorem 12 (Lower Bound for XQuery). There is an XQuery 
query Q such that the problem of evaluating Q on an input XML 
document stream of length N does not belong to the class LasVe- 
gas-RST(o(log JV) , 0{<ffi / log iV) , 0( 1 ) ). H 

Theorem 13 (Lower Bound for XPath). There is an XPath query 
Q such that the problem of filtering an input XML document stream 
with Q (i.e., checking whether at least one node of the document 
matches the query) does not belong to the class co-RST(o(logA'), 
0«fN/\agN),0{\)). " H 

For proving the Theorems [12] and Q~3] we represent an instance 
xi#- ■ -#x m #yi ■ ■■#y m # of the SET-EQUALITY problem by an XML 
document of the form 

<instance> 
<setl> 

<item> <string> x\ </string> </item> 

<item> <string> x m </string> </item> 
</setl> 

<set2> 

<item> <string> y\ </string> </item> 

<item> <string> y m </string> </item> 
</set2> 
</instance> 

(For technical reasons, we enclose every string x,- and yj by a 
string-element and an item-element. For the proof of Theo- 
rem Q21 one of the two would suffice, but for the proof of Theo- 



rem Q~3] it is more convenient if each Xj and yj is enclosed by two 
element nodes.) 

It should be clear that, given as input x\#- ■ ■ #x m #y\ ■ ■ ■ #y m #, the 
above XML document can be produced by using a constant num- 
ber of sequential scans, constant internal memory space, and two 
external memory tapes. 

Proof of Theoremll2l 

The SET-EQUALITY problem can be expressed by the following 
XQuery query Q := 

<result> 

if ( every $x in /instance/setl/item/string satisfies 
some $y in /instance/set2/item/string satisfies 
$x = $y ) 

and 

( every $y in /instance/set2/item/string satisfies 
some Sx in /instance/setl/item/string satisfies 
$x = $y ) 
then <true/> 
else (} 
</result> 

Note that if {x\ , . . ,x m } = {yi , . . ,y m }, then Q returns the document 
<result><true/x/result>, and otherwise Q returns the 
"empty" document <result></ resultx Thus, if Q could be 
evaluated in LasVegas-RST(o{logN),0(^N/logN),0(l)), then 
the SET-EQUALITY problem could be solved in RST(o(logiV), 
0{\fN/\ogN), 0(1)), contradicting Theorem© □ 

Proof of Theorem[13l 

The XPath query Q of FigureQ]selects all item-nodes below set 1 
whose string content does not occur as the string content of some 
item-node below set2 (recall the "existential" semantics of 
XPath [20 1). In other words: Q selects all (nodes that represent) 
elements in X — Y , for X := {x\ , . . ,x,„} and Y := {y\ , . . ,y m }- 

Now assume, for contradiction, that the problem of filtering an 
input XML document stream with the XPath query Q (i.e., check- 
ing whether at least one document node is selected by Q) belongs 
to the class co-RST(o(logA ? ), 0{\fN/logN), 0(1)). Then, clearly, 
there exists an (o(logA'),0(v / A'/logA'),0(l))-bounded random- 
ized Turing machine T which has the following properties for ev- 
ery input x\#- ■ -#x m #yi#- ■ ■ #y m # (where X := {x\ , . . ,x m } andT := 
{yi) ••,?«}): 

(1) If Q selects at least one node (i.e., X — Y^(b, i.e., X % Y), then 
T accepts with probability 1 . 

(2) If Q does not select any node (i.e., X- Y = 0, i.e., X C Y), then 
T rejects with probability > 0.5. 

This machine T can be used to solve the SET-EQUALITY problem 
by a machine T as follows: First, T starts T with input x\#- ■ ■ #x m # 
yi#- ■ - #y m #. Afterwards, T starts T with input y\#- ■ -#y m #x\#- ■ ■# 
x m #. If both runs reject, then T accepts its entire input. Otherwise, 
T rejects. 

Let us analyze the acceptance/rejectance probabilities of T: 

(i) lfX^Y, then either X % Y or Y % X, and thus, due to (1), at 
least one of the two runs of T has to accept. The machine T 
will therefore reject with probability 1. 

(ii) If X = Y, then X C Y and Y C X. Due to (2), we therefore know 
that each of the two runs of T will accept with probability > 0.5 
and thus, in total, T will accept with probability > 0.25. 

To increase the acceptance probability to 0.5, we can start two inde- 
pendent runs of T and accept if at least one of the two runs accept. 
In total, this leads to a (o(logiV), 0(\/N/\ogN), 0(l))-bounded 
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descendant: : s e 1 1 / child: : i t em [ not child: : s t r i n g = 

ancestor::instance /child::set2 /child::item/child::string ] 



Figure 1: The XPath query Q used in the proof of Theorem H3l 



randomized Turing machine which, on every input xi#- • -#x m # 
y\#---#y m #, 

- accepts with probability > 0.5, if {x\ , . . ,x m } = {y\ , . . ,y m }, 

- rejects with probability 1, otherwise. 

In other words: This machine shows that the SET-EQUALITY prob- 
lem belongs to RST(o(logiV), 0(\fN/\ogN), 0(1)), contradicting 
Theorem[6] Therefore, the problem of filtering an input XML doc- 
ument stream with the XPath query Q does not belong to the class 
co-RST(o(logAO,0(v / iV/logiV),0(l)). ' □ 

5. LIST MACHINES 

This section as well as the subsequent sections are devoted to 
the proof of Theorem [6] For proving Theorem [6] we use list ma- 
chines. The important advantage that these list machines have over 
the original Turing machines is that they make it fairly easy to track 
the "flow of information" during a computation. 

In 1101 we introduced the notion of deterministic list machines 
with output. In what follows, we propose a nondeterministic ver- 
sion of such machines without output, i.e., nondeterministic list 
machines for solving decision problems. To introduce nondeter- 
minism to the notion of 1 10 1 requires some care — the straightfor- 
ward approach where, instead of the transition functions used in 
1101 . transition relations are allowed, will lead to a machine model 
that is too weak for adequately simulating nondeterministic Tur- 
ing machines. Therefore, instead of using transition relations, the 
following notion of nondeterministic list machines allows explicit 
nondeterministic choices in transitions. 

Definition 14 (Nondeterministic List Machine). 

A nondeterministic list machine (NLM) is a tuple 

M = (t,m,I,C,A,a Q ,a,B,B a cc) 

consisting of 

- a t £ N, the number of lists. 

- an m £ N, the length of the input. 

- a finite set / whose elements are called input numbers (usu- 
ally, / C N or / C {0, 1}*). 

- a finite set C whose elements are called nondeterministic 
choices. 

- a finite set A whose elements are called (abstract) states. 
We assume that /, C, and A are pairwise disjoint and do not 
contain the two special symbols '{' and ')'. 

We call A:=/UCUAU{{,)} the alphabet of the machine. 

- an initial state ao £ A. 

- a transition function 

a : (A\B) x (A*)' xC -> (A x Movement') 

with Movement := { (head-direction, move) | 

head-direction £ { — 1 , + 1 } , 

move £ {true,false} }. 

- a set B C A of final states. 



- a set B acc C B of accepting states. (We use B rej := B \ B acc to 
denote the set of rejecting states.) H 

Intuitively, an NLM M = (t,m,I,C,A,OQ, a,B,B acc ) operates as 
follows: The input is a sequence (vi , . . . , v m ) £ I m . Instead of tapes 
(as a Turing machine), an NLM operates on t lists. In particular, 
this means that a new list cell can be inserted between two existing 
cells. As for tapes, there is a read-write head operating on each list. 
Cells of the lists store strings in A* (and not just symbols from A). 
Initially, the first list, called the input list, contains (vi , . . . , v m ), and 
all other lists are empty. The heads are on the left end of the lists. 
The transition function only determines the NLM's new state and 
the head movements, and not what is written into the list cells. In 
each step of the computation, the heads move according to the tran- 
sition function, by choosing "nondeterministically" an arbitrary el- 
ement in C. In each computation step, the current state, the content 
of all current head positions, and the nondeterministic choice c £ C 
used in the current transition, are written behind each head. When 
a final state is reached, the machine stops. If this final state belongs 
to B acc , the according run is accepting; otherwise it is rejecting. 
Figure [2] illustrates a transition of an NLM. The formal definition 
of the semantics of nondeterministic list machines can be found in 
Appendix [51 



Figure 2: A transition of an NLM. The example transition is 
of the form (a,X4,y2,Z3,c) — > [b,(— 1, false), (1, true), (1, false)). 
The new string w that is written in the tape cells consists of 
the current state a, the content of the list cells read before the 
transition, and the nondeterministic choice c. 

An NLM is called deterministic if |C| = 1. 
For every run p of an NLM M and for each list x of M, we define 
rev(p,T) to be the number of changes of the direction of the T- 
th list's head in run p. We say that M is (rj)-bounded, for some 
r,t £ N, if it has at most t lists, every run p of M is finite, and 
l + i;' T=1 rev(p,T) < r. 

Randomized list machines are defined in a similar way as ran- 
domized Turing machines: For configurations y and / of an NLM 
M, the probability Pr(y — y 1 ) that y yields / in one step, is de- 
fined as |{c £ C : Y is the c-successor of y}|/|C|. For a run p = 
(pi , . - ,pi), the probability Pr(p) that M performs run p is the prod- 
uct of the probabilities Pr(p,- -^m Pi+1 ), f° r all ' < For an input 
v £ /"', the probability that M accepts v is defined as the sum of 
Pr(p) for all accepting runs p of M on input v. 
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The following notation will be very convenient: 

Definition 15 (Pm(v, c)). Let M be an NLM and let £ £ N such that 
every run of M has length < £. For every input v £ V and every 
sequence c = (c'i,..,q) 6 C , we use Pm(v,c) to denote the run 
(pi , . . , p,t) obtained by starting M with input v and by making in its 
i-th step the nondeterministic choice c,- (i.e., p I+ i is the c,-successor 
of Pi). H 

6. LIST MACHINES CAN SIMULATE 
TURING MACHINES 

An important property of list machines is that they can simulate 
Turing machines in the following sense: 

Lemma 16 (Simulation Lemma). Let r,j:N^N,feN, and let 
T = (Q,Y,,A,qo,F,F a cc) be an (r,s,t)-bounded NTM with a total 
number of t+u tapes and with {□,#} CE. Then for every m,n £ N 
there exists an (r(m-(n+l)),t) -bounded NLM M mn = M = (t,m,I, 
C,A,a ,a,B,B acc ) with I = (£\ {□,#})" and \C\ < 2°M m '("+ 1 )» 
where £(N) is an upper bound on the length of T 's runs on input 
words of length N, and 



\A\ < 2 



d-t 2 -r{m-(n+l))-s{m-(n+l)} + 3«-log(m-(n+l)) 



(2) 



for some number d = d(u, |2| , |£|) that does not depend on r, m, n, 
t, such that for allv = (vi , . . , v m ) £ F" we have 

Pr(M accepts v) = Pr(T accepts v\#- ■ ■ v m #) . 

Furthermore, if T is deterministic, then M is deterministic, too. H 

In Section [8] we will use the simulation lemma to transfer the 
lower bound results for list machines to lower bound results for 
Turing machines. 

In [10 1, the simulation lemma has been stated and proved for 
deterministic machines. For nondeterministic machines, the con- 
struction is based on the same idea. However, some further work is 
necessary to assure that the according list machine accepts with the 
same probability as the given Turing machine. Throughout the re- 
mainder of this section, the proof idea is given; a detailed proof of 
Lemma [T6l can be found in Appendix Icl For proving Lemma [T6l 
the following straightforward characterization of probabilities for 
Turing machines is very convenient. 

Definition 17 (C T and p T (w,c)). Let T be an NTM for which 
there exists a function I : N — > N such that every run of T on a length 
N input word has length at most £(N). Let b := max{|Next7-(y)| : 
7 is a configuration of T} be the maximum branching degree of T 
(note that b is finite since 7"s transition relation is finite). Let b' := 
lcm{l, . . ,b} be the least common multiple of the numbers 1,2,. . ,b, 
and let Cj :={ 1 ,..,£>'} . For every N £ N, every input word w £ E* 
of length N, and every sequence c = (c\ , ■ ■ ■ , c^ N ^ ) £ [CtY . we 
define p T (w,c) to be the run (pi,..,pj.) of T that is obtained by 
starting T with input w and by choosing in its i-th computation step 
the (cj mod |Next7-(p,)|)-th of the |Nextr(p;)| possible next con- 
figurations. H 

Lemma 18. Let T be an NTM for which there exists a function £ : 
N — ♦ N such that every run ofT on a length N input word has length 
at most £{N), and let Cj be chosen according to Definition]! 7\ Then 
we have for every run pofTon an input w of length N that 



\{ce(C T )<W : p T (w.c)=p}\ 
|(C r )'W| 



and, in total, 



Hp) 

Pr(T accepts w) 
The (straightforward) proof can be found in Appendix IAI 



\{ce{C T )' [N) : Pt(w-c) accepts}] 



For proving Lemma[T6l let T be an NTM. We construct an NLM 
M that simulates T. The lists of M represent the external mem- 
ory tapes of T . More precisely, the cells of the lists of M repre- 
sent segments, or blocks, of the corresponding external memory 
tapes of T in such a way that the content of a block at any step 
of the computation can be reconstructed from the content of the 
cell representing it. The blocks evolve dynamically in a way that 
is described below. M's set C of nondeterministic choices is de- 
fined as C := (Ct) , where Cj is chosen according to Definition! 171 
and I := £{m ■ (n + 1)) is an upper bound on 7"s running time and 
tape length, obtained from Lemma[3] Each step of the list machine 
corresponds to the sequence of Turing machine steps that are per- 
formed by T while none of its external memory tape heads changes 
its direction or leaves its current tape block. Of course, the length 
£' of this sequence of 7"s steps is bounded by 7"s entire running 
time I. Thus, if c = {c\, . . , eg) £ C = (Ct) 1 is the nondeterministic 
choice used in M's current step, the prefix of length £' of c tells 
us, which nondeterministic choices (in the sense of Definition! 17b 
T makes throughout the corresponding sequence of £' steps. The 
states of M encode: 

- The current state of the Turing machine T. 

- The content and the head positions of the internal memory tapes 

f+l,...,f + KOf T, 

- The head positions of the external memory tapes 1, .. . ,t. 



For each of the external memory tapes 1, 
the block in which the head currently is. 



,t, the boundaries of 



Representing T's current state and the content and head positions 
of the u internal memory tapes requires \Q\ ■ 2°( s ( m '(' ! + 1 ')) . s(m ■ 
(n+ 1))" states. The / head positions of the external memory tapes 
increase the number of states by a factor of £' . The 2t block bound- 
aries increase the number of states by another factor of £ 2 ' . So over- 
all, the number of states is bounded by \Q\ ■ 2°( S M"+ 1 ))) - s {nf (« + 
1))" • £ 3 '. By Lemma[3] this yields the bound ©. 

Initially, for an input word v\#-- ■ v m #, the first Turing machine 
tape is split into m blocks which contain the input segments v;# 
(for 1 < i < m), respectively, v m #n^" +1 ) (that is, the m-th in- 
put segment is padded by as many blank symbols as the Turing 
machine may enter throughout its computation). All other tapes 
just consist of one block which contains the blank string □ . The 
heads in the initial configuration of M are on the first cells of their 
lists. Now we start the simulation: For a particular nondetermin- 
istic choice c\ = (e\\,c\2,..,c\i) £ C = (Ct) I , we start T's run 
Pr( v l#'" ! c ll c l2 c l3 " ' )• As long as no head of the external mem- 
ory tapes of T changes its direction or crosses the boundaries of 
its current block, M does not do anything. If a head on a tape 
('o £ {1, . .. ,t} crosses the boundaries of its block, the head i'o of M 
moves to the next cell, and the previous cell is overwritten with suf- 
ficient information so that if it is visited again later, the content of 
the corresponding block of tape io of T can be reconstructed. The 
blocks on all other tapes are split behind the current head position 
("behind" is defined relative to the current direction in which the 
head moves). A new cell is inserted into the lists behind the head, 
this cell represents the newly created tape block that is behind the 
head. The newly created block starting with the current head posi- 
tion is represented by the (old) cell on which the head still stands. 
The case that a head on a tape i'o £ {1, . . . ,t} changes its direction 
is treated similarly. 

The simulation stops as soon as T has reached a final state; and 
M accepts if, and only if, T does. A close look at the possible runs 
of T and M shows that M has the same acceptance probabilities as 
T. 
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7. LOWER BOUNDS FOR LIST MACHINES 

This section's main result is that it provides constraints on a list 
machine's parameters, which ensure that list machines which com- 
ply to these constraints can neither solve the multiset equality prob- 
lem nor the checksort problem. In fact, we can show a slightly 
stronger result, the precise formulation of which requires the fol- 
lowing observation. 

Definition 19 (sortedness). Let m £ N and let n be a permutation 
of {l,..,m}. We define sortedness (n) to be the length of the 
longest subsequenc^l of ( jt( 1 ) , . . , 7z(m)) that is sorted in either as- 
cending or descending order (i.e., that is a subsequence of (1, . . ,m) 
or of (m, . . , 1)). H 

Remark 20. It is well-known that for every permutation n of 
{ 1 , . . , m}, sortedness(n) £ Q.(^/m) , and that there exists a partic- 
ular permutation q> := q>„, with sortedness((p) < 2^/m — 1 . In fact, 
one way of finding such a permutation is to let (<p(l), . . , <p(m)) 
be the numbers 1 , . . , m, sorted lexicographically by their reverse 
binary representation. H 

Lemma 21 (Lower Bound for List Machines). 

Let k,m,n,r,t £ N such that m is a power of 2 and t > 2, m > 
24 • {t+l) 4r +1, k > 2m + 3, n > 1 + (m 2 + 1) ■ log(2£). We 
let I := {0,1}", identify I with the set {0, 1 , . . , 2 n — 1}, and divide it 
into m consecutive intervals I\ , . . ,I m , each of length 2" jm. Let <p 
be a permutation of {I,.. ,m\ with sorted?iess((p) < 2i/m— 1, and 
let J := l^r) x • • • x /,p (m ) x l\ X •• • X I m . 

Then there is no (r,t)-bounded NLM M = (t,2m,I,C,A,ao,a,B, 
B a cc) with \A\ < k and I — {0, 1}", such that for all V = . . , v m , 
v[,. .,v' m ) £ J? we have: 

If (vi,..,v m ) = (v' 9(1) ,..,v' 9(m) ), then Pr(M accepts v) > \; 
otherwise Pr(M accepts v) — 0. H 

It is straightforward to see that the above lemma, in particular, 
implies that neither the (multi)set equality problem nor the check- 
sort problem can be solved by list machines with the according 
parameters. 

The proof of Lemma l2Tlis based on the following ideas (the de- 
tailed proof is given in Appendix fDt: 

1. Suppose for contradiction that M is an NLM that meets the 
lemma's requirements. 

2. Observe that there exists an upper bound I on the length of M's 
runs (Lemma [3"Tlllal> in Appendix [DJ and a particular sequence 
c = (c\ , . . , q) 6 C of nondeterministic choices (Lemma I26t . 
such that for at least half of the inputs v : = ( v i , . . , v m , v'j , . . , vj fl ) £ 
J with (vi , . . , v m ) = (v/, (1) , . . , v'p (m) ), the particular run p M (v,c) 
accepts. 

We let J'accx :={v£/: Pm{v,c) accepts} and, from now on, 
we only consider runs that are generated by the fixed sequence 
c of nondeterministic choices. 

3. Show that, throughout its computation, M can "mix" the rela- 
tive order of its input values only to a rather limited extent (cf., 
Lemma[37t. This can be used to show that for every run of M on 
every input (vi , . - , v m , v'j, . . , V m ) £ J? there must be an index ;'q 
such that \>i and v^/^ are never compared throughout this run. 

4. Use the notion of the skeleton of a run (cf., Definitionl28l>. which, 
roughly speaking, is obtained from a run by replacing every in- 

2 A sequence (si , . . ,s^) is a subsequence of a sequence (i-'j , . . ,Sj,), 
if there exist indices ji < ■ ■ ■ < jx such that s\ = s'^ , S2 = s'^ , . . . , 



put value v, with its index i and by replacing every nondetermin- 
istic choice ceC with the wildcard symbol "?". In particular, 
the skeleton contains input positions rather than concrete input 
values; but given the skeleton together with the concrete input 
values and the sequence of nondeterministic choices, the entire 
run of M can be reconstructed. 

5. Now choose £ to be a skeleton that is generated by the run 
Pm(v,c) for as many input instances v £ J^acc,c as possible, and 
use y acc c r to denote the set of all those input instances. 

6. Due to[5] there must be an index ;'o such that for all inputs from 
■^acc c f> me va lu es v ! an d v '^(, ) (i- e -> me values from the input 
positions i'o and m + <p(io)) are never compared throughout the 
run that has skeleton To simplify notation let us henceforth 
assume without loss of generality that (n = 1. 

7. Now fix (v2 , . . , v m ) such that the number of vi with 

V{n) ■= (Vl , V 2) •-, V m , Vy-i (j),.. ,V v -i( m )) £ J^acc^r 
is as large as possible. 

8. Argue that, for our fixed (v2, • • , v m ), there must be at least two 
distinct vi and w\ such that V{v{) £ ^ arC e^ and 
V(wi) £ :J? acC c r- This is achieved by observing that the num- 
ber of skeletons depends on the machine's parameters t, r, m, k, 
but not on n (Lemma [32l and by using the lemma's assumption 
on the machine's parameters t,r,m,k,n. 

9. Now we know that the input values of V(v{) and V(wi) co- 
incide on all input positions except 1 and m+<p(l). From [3] 
we know that the values from the positions 1 and m+<p(l) are 
never compared throughout M's (accepting) runs p^(l/(vi),c) 
and Pm{V (w\),c) . From this we obtain (cf., Lemma [34l an ac- 
cepting run Pm(u,c) of M on input 

U '.= (u\, • • ,W m ,ZYp . . ,M m ) 

:= ( Vl , V 2 , . . , V,„ , W^-i (!) , Wp-l (2 ) , • • , (m ) ) . 

In particular, this implies that Pr(M accepts u) > 0. However, 
for this particular input u we know that u\ = vi 7^ w\ = u^y 
and therefore, {u\,..,u m ) ^ Wa(iy^a( m ))- This gives us a 
contradiction to the assumption that Pr(M accepts v) = for 
all inputs v= (vi , . . , v m , v', , . . , v^) with (vi,..,v m ) 7^ (v^,.., 

8. LOWER BOUNDS FOR 
TURING MACHINES 

Lemma 22. Let r, s : N — > N such that r(N) = o(logiV) and s(N) = 
o(\/N/r(N)). Then, there is no (r,s,0(l))-bounded {\,0)-RTM 
that solves the following problem CHECK-<p. H 



CHECK-(p 
Instance: v\ # . . . v m #v'j# . . . V m #, 

where m > is a power of 2, and 
(v 1 ,...,v m ,v' 1 ,...,v,' fI ) £/ 9( i) x ••• x/,p (m) xA x 
■•• x I m , where (p := <p m is the permutation of 
{l,..,m} obtained from Remark l20l and the sets 
l\,..,l m C {0, 1 y are obtained as the partition of 
the set {0, 1}'" into m consecutive subsets, each of 
size2( m3 Vm. 

Problem: Decide if {v u . . . ,v m ) = (i/, (1) , . . . , i/, (jf>) ) . 
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Proof: Suppose for contradiction that there is a t 6 N such that the 
problem CHECK-<p is solved by an (r,s,t) -bounded (2,0)-RTM T. 
Without loss of generality we may assume that t > 2. 

Let d be the constant introduced in Lemma [T6] (the simulation 
lemma). Let m be a sufficiently large power of 2 such that 

m > 24-(f+l) 4 ' r ( 2m > 3+1 » + l and (3) 

n? > 1+ d-t 2 -r(2m- (m 3 + l)) ■ s(2m- (m 3 + l)) (4) 
+ 3f -log(2m- (m 3 + l)). 

Such an m exists because r(N) = o(logA') (for Equation l[3j) and 
r(N) ■ s{N) = o (s/~N) (for Equation ©). Let n := m 3 and / := 
{0,1}". By LemmafTol there is an (r(2m-(n+l)),f) -bounded NLM 

M = (t,2m,I,C,A,ao,a,B,B acc ) with 

^ <- 2 fi,, ' 2,r ( 2m, ("+ 1 )) ,J ( 2 " ,, ( ,! + 1 )) + 3f-log(2m-(n+l)) 

states that simulates T on inputs from l 2m . In particular, for all 
instances 

V = (vi,..,v m ,v[,..,v' m ) e x ■■• x/ 9(m) X/] X ••■ x/ m 
the following is true: 

(*): rf(v 1; ..,v m ) = (v^ j j,..,!'^^), then Pr(M accepts v)>i; 
otherwise Pr(M accepts v) = 0. 

We clearly have k > 2m + 3. By l(3j, we have 

m > 24-(?+l) 4 ' r ( 2m '( ffl3+1 » + l = 24-(f+l) 4 ''' (2 " ,,( ' ,+1), + l. 

By 10, we have 

3 

n — m 

> l + (m 2 + l)- (1+ d-t 2 -r(2m-(m i + l))-s{2m-(m 3 + l)) 
+ 3f-log(2ra- (m 3 + l))) 

= l + (m 2 + l)- (1+ d-t 2 -r(2m-(n+l))-s(2m-(n+l)) 
+ 3f-log(2m-(n+l))) 

> l + (m 2 + l)-(log(2)+log(fc)) 

= l + (m 2 + l)-log(2F). 

Thus, (*) is a contradiction to Lemma |2T] and the proof of 
Lemmal22lis complete. □ 

Proof of Theorem|6j For inputs that are instances of the problem 
CHECK-<p, the problems SET-EQUALITY, MULTISET-EQUALITY, 
CHECK-SORT, and CHECK-<p coincide. Thus, Lemma[22limmedi- 
ately implies Theorem[6] □ 

9. CONCLUSION 

We have proved tight lower bounds for the natural decision prob- 
lems ( multiset equality and checksort, in our Turing machine based 
computation model for processing large data sets. These lower 
bounds do not only hold for deterministic, but even for randomized 
algorithms with one-sided bounded error probability. Our results 
are obtained by carefully analyzing the flow of information in a 
Turing machine computation. 

As applications of these lower bound results, we obtained lower 
bounds on the worst case data complexity of query evaluation for 
the languages XQuery, XPath, and relational algebra on data 
streams. 

We complement our lower bounds for checksort and (multi)set 
equality by proving that these problems can be solved by nondeter- 
ministic machines and by randomized machines with complemen- 
tary one-sided error probabilities. As a consequence, we obtain 



a separation between the deterministic, the randomized, the co- 
randomized, and the nondeterministic external memory complexity 
classes. 

A specific problem for which we could not prove lower bounds, 
even though it looks very similar to the set equality problem, is the 
disjoint sets problem, which asks if two given sets of strings are 
disjoint. Another important future task is to develop techniques for 
proving lower bounds (a) for randomized computations with two- 
sided bounded error and (b) for appropriate problems in a setting 
where fl(logA') head reversals (i.e., sequential scans of external 
memory devices) are available. 
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APPENDIX 



A. TURING MACHINE BASICS 

Definition 23 (Notation concerning Turing machines). 

Let T = {Q,~L,A.,qQ,F,F acc ) be a nondeterministic Turing machine (NTM, for short) with t+u tapes, where 
Q is the state space, E the alphabet, qo 6 Q the start state, FCg the set of final states, F acc C F the set of 
accepting states, and 

A C (Q\F)xU+" xQxl! +u x{L,N,R}' +u 

the transition relation. Here L,N,R are special symbols indicating the head movements. 

We assume that all tapes are one-sided infinite and have cells numbered 1,2,3, etc, and that □ £ E is the 
"blank" symbol which, at the beginning of the TM's computation, is the inscription of all empty tape cells. 

A configuration of T is a tuple 

(q,p h ...,p t+u ,w h ...,w t+u ) € <2xN'+"x(E*)'+", 

where q is the current state, p\,..., p t+u are the positions of the heads on the tapes, and w\ , . . . , w t + u are the 
contents of the tapes. For a configuration y we write Nextr(y) for the set of all configurations / that can be 
reached from y in a single computation step. 

A configuration is called final (resp., accepting) if its current state q is final (resp., accepting), that is, q £ F 
(resp., q £ F acc ). Note that a final configuration does not have a successor configuration. 

A run of T is a sequence p = {Pj)jeJ of configurations Pj satisfying the obvious requirements. We are 
only interested in finite runs here, where the index set / is { 1, . . . , £} for an I £ N, and where pi is final. 

When considering decision problems, a run p is called accepting (resp., rejecting) if its final configuration 
is accepting (resp., rejecting). When considering, instead, Turing machines that produce an output, we say that 
a run p outputs the word w' if p end in an accepting state and vJ is the inscription of the last (i.e., f-th) external 
memory tape. If p ends in a rejecting state, we say that p outputs "I don't know" . 

Without loss of generality we assume that our Turing machines are normalized in such a way that in each 
step at most one of its heads moves to the left or to the right. H 



Proof of LemmalTsl 

To prove (a), observe that for every run p = (pi , . .,Pk) of T on w we have k < £(N), and 

\{ceC< T {N> :p T (w.c)=p}\ _ 1 ( o \C T \ \ , lc MN)-{k-l) _ T~f 1 _ p r ( o) 

|4 (A,) | ~ icfl UA|N«tr(A)| I - JLl |Natr(A)| ~ ' 

(b) follows directly from (a), since 

n frp . s def v , , (a) ^ \{ceCj {N) :pr(w,c) = p}| 
Pr(J accepts w) = ^ Pr (P) = L tW) 

P :pi S ,u, a cc C pU„g ; | 



|{c £ Cj' : Pt(w,c) accepts}| 



p7~ (n'.r) accepts 



B. FORMAL DEFINITION OF THE SEMANTICS OF NONDETERMIN- 
ISTIC LIST MACHINES 

Formally, the semantics of nondeterministic list machines are defined as follows: 
Definition 24 (Semantics of NLMs). 



(a) A configuration of an NLM M = (t,m,I,C,A,ao,a,B,B acc ) is a tuple (a,p,d,X) with 



W 



e N' , d : 



e {-1.+1}' , X: 



where N = {1,2,3,..} is the set of positive integers, 

- a 6 A is the current state, 

- ~p is the tuple of /;ea<i positions, 
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- d is the tuple of head directions, 

- Xj = (xji,. . . ,x im .) <E (A*) m ' for some m,- > 1, contains the content of the cells. (The string Xij € A* 
is the content of the jth cell of the rth list.) 

(b) The initial configuration for input (vi, . . . ,v m ) <E I" 1 is a tuple (a,p,d,X), where a = ao,p — (1, . . . , 1) T , 
d= (+1,..., + 1) T , undX = (x u ...,x,) T with 

xi = ((vi>,...,(v m » € (A*)" 

andjC2 = -"=x, = ((>) € (A*) 1 . 

(c) For a nondeterministic choice c e C, the c-successor of a configuration (a,p,d,X) is the configuration 
(a 1 ,p' ,d ,X') defined as follows: Suppose that 

a(a,xi iPl ,..,x t ,p„c) = (b,ei,...,e t ). 

We let a' = b. For 1 < i < t , let m,- be the length of the list x t , and let 

{(—IJalse) if pi = 1 and e,- = (— I, true), 
{+\,false) if pi = nti and e, = (+l,true), 
ej otherwise. 

This will prevent the machine from "falling off" the left or right end of a list. I.e., if the head is standing 
on the rightmost (resp., leftmost) list cell, it will stay there instead of moving a further step to the right 
(resp., to the left). 

We fix fi e {0, 1} such that /,- = 1 iff (move; = true or head- direction ,- 7^ d,) . 

If fi = for all i <E {1, . . ,t}, then we let p' := p, d' := d, and X' := X (i.e., if none of the machine's head 
moves, then the state is the only thing that may change in the machine's current step). 
So suppose that there is at least one ( such that f 7^ 0. In this case, we let 

y ■= a{x\, Pl )---{xt, Pl ) (c)- 

For all i <E { 1 ,..,«}, we let 

{{xi,i , ■ ■ ■ , Xi, Pl -l, y, Xi. Pi+ \, x im ) if move; = true, 

(■*/, 1 ,---,Xi, Pi - 1 1 y, Xi, Pp x itPi+ i , x im ) if di = + 1 and move,- = /a/se , 
1 ,-•-,•«(>- 1 , Xi, Pi , y, x i}Pi+ \ , xi m ) if d, 1 = - 1 and move,- = /afee , 

and, finally, 

{p, + l if ej = (+l,frae), 
Pi — I if ej- = (— 1, frae) , 
+ l ife; = (+l,/afae), 
p,- if ej. = (-IJalse). 

(d) A configuration (a,~p,d,X) is final (accepting, resp., rejecting), if a e 6 (B flcc , resp., B rej - := B\B acc ). 

A (finite) ran of the machine is a sequence (pi ,pf) of configurations, where pi is the initial config- 
uration for some input, p( is final, and for every i < £ there is a nondeterministic choice c,- 6 C such that 
p,- + i is the c,-successor of p,-. 

A run is called accepting (resp., rejecting) if its final configuration is accepting (resp., rejecting). 

(e) An input (vi , . . ,v m ) <E I m is accepted by machine M if there is at least one accepting run of M on input 

(vi,..,v m ). H 



It is straightforward to see that 

Lemma 25. Let M = (t,m,I,C,A,ao, a,B,B acc ) be anNLM, and let £ be an upper bound on the length ofM's 
runs. 

(a) For every run pofM on an input v € I m , we have 

Vr(n , _ \{ceC f :p M (v,c)=p}\ 
np; " \C*\ 

(b) 

\{c £ C ( : Pm(v,c) accepts}\ 



Pr(M accepts v) 



|C*| 
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Proof: To prove (a), observe that for every run p = (pi , . . , p^.) of M on v we have k< £, and 



\{ceC>:pM<J'.c)=p}\ = ^.|pj| {ceC :p i+ iisthe C -successorofp ; }|).|C| £ -(' " 



(=l 

(b) follows directly from (a), since 



k-l 

Y[Vi(Pi^Mpi+l) = Pr(p) 



. _, def _ , v (a) ^ |{c£C f : p M (v,c)= p}| 

Pr(M accepts v) = ^ Pr (P) = L 



1 | {c £ C e : p(v,c) accepts} | 



p M (v,c) accepts 



C. PROOF OF THE SIMULATION LEMMA 

Let T = (Q,T.,A,qo,F,F aC c) be the given (r, i,?)-bounded nondeterministic Turing machine with t + u tapes, 
where the tapes 1 , . . . , t are the external memory tapes and tapes t + l,...,t + u are the internal memory 
tapes. Let m,n6N and N = m- (n+1). Every tuple v = (vi,..,v m ) £ /'" corresponds to an input string 
v := V] #V2#--v m # of length N. Let r := r(N) and s := s(N). 

By Lemma[3] there is a constant c\ = c\ (u, \Q\ , |E|), which does not depend on r, m, n, t, such that every 
run of T on every input v, for any v £ I m , has length at most 

£(N) := N-2 Cl - r < t+s ^ (5) 
and throughout each such run, each of 7"s external memory tapes 1, . . . ,t has length < £(N). We let t := £(N). 

Step 1: Definition ofM's set C of nondeterministic choices. 

M's set C of nondeterministic choices is chosen as C := {Cry, where Cj is chosen according to Definition! 171 

H 



Step 2: Definition of a superset A of M's state set A. 

Let Q be the set of potential configurations of tapes f+1, . . . ,t+u, together with the current state of T, that is, 



Q { {q,Pt+\, ■ ■ ■ ,Pt+u,w t+ \, . . . ,w, +u ) 



q£Q, pt+t e {l, •-•,*}> 

w t+i eE^(foraUie{l,...,«}) 
Then for a suitable constant ci = C2(u,\Q\, |L|) we have 

\Q\ < 2 e ^. (6) 

We let 

A := | (g, p l , . . .p,) | q e Q, and for each j 6 {1, . . ,f}, 

= (p!,p!-,p!-, head-direction j) with 

♦ 

p j £ { 1 ,..,£} , head-direction j £ {+1,-1}, and 
either /n = pj = 0, 

or pjj.pj €{!,..,£} with < pt < pl 1 . 

Here, is a symbol for indicating that p? and pj- are "undefined", that is, that they cannot be interpreted as 
positions on one of the Turing machine's external memory tapes. 

Later, at the end of Step 4, we will specify, which particular subset of A will be designated as M's state set 
A. With any choice of A as a subset of A we will have 

|A| < \A\ < \Q\-(£+l) 3 '' -2' < 2 C2 '' 5 -(iV-2 Cl ' r '('+' 5 ) + l) 3 ''-2 f < 2^' 2 '" 
for a suitable constant d = d(u, \Q\, This completes Step 2. H 
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Step 3: Definition ofM's initial state oq and M's sets B and B acc of final states and accepting states, respec- 
tively. 

Let 

q := (<?o,l^,nV^) 

It u 

be the part of 7"s initial configuration that describes the (start) state qo of T and the head positions and initial 

(i.e., empty) content of the tapes f+1, . .,t+u (that is, the tapes that represent internal memory). 

Let 

(I T ]] l , ,■ *■ \ / (M,«+l,+l) ifm>l 
Pi {p\,p\,p\Mad-directioni) := j + if m = 

and, for all <E {2, ..,?}, 

— [fin 

Pi : ~ (p i,p-, Pi, head-direction i) := (1,1, £,+l). 
As start state of the NLM M we choose 

ao := {qo,P\,P2,--:Pt) 

As M's sets of final, resp., accepting states we choose B := BOA, resp., 6 aee := B acc flA with 

S := {{q,p x ,p2---,p t )<EA \ q is of the form (q,p, y) e Q for some q e F] 

Bacc := {(q,Pi,P2,--,Pt)€A \ q is of the form (q,p,y) G Q for some q e F acc }. 

I.e., a state of M is /iraZ (resp., accepting) if, and only if, the associated state of the Turing machine T is. This 
completes Step 3. H 



Step 4: Definition of M's transition function a : (A \B) x (A*)' x C — » (A x Movement') . 
We let 

Con/ r := | q£Q, and 

for all je {l,..,f+M}, p ; eN, 

for all ; e {1, . . ,?}, w ; - £ {©}*£*{©}* with w J>p . € E, 
forally€{l,..,«}, }, 

where © is a symbol nof in L, and denotes the py-th letter in the string wj. 

Intended meaning: The symbol © is used as a wildcard symbol that may be interpreted by any symbol in 
E. An element in Conf T gives (potentially) incomplete information on a configuration of T, where the contents 
of tapes l,..,t might be described only in some part (namely, in the part containing no ©-symbols). 

We let A := / U C UA U {{, ) }. By induction on i we fix, for i > 0, 

- a set Ai C A 

- asetK; C (A\B) x (A*)', 

- a set Li C A*, letting 

L t := { a(yi) ■■■(};} (c) : (a,yi, . . ,y t ) € K t and c € C } (7) 

- a function 

configj : Ki -» Con/ r U {±} 

Intended meaning: When the NLM M is in a situation K e Ki, then config^ K) is the Turing machine's con- 
figuration at the beginning of M's current step. If configj(K) = ±, then K does not represent a configuration 
of the Turing machine. 

- the transition function a of M, restricted to Ki, that is, 

OL\ K . : Ki x C — > A x Movement' 

- for every tape y € {1, ..,*}, a function 



tape-configji ■ U ^ { (w,P ][ ,P 11 



either 1 < pU < pJI < £ and 

W e {®}P ll - 1 U' 1 -P ll+1 {®}^ N 1-P 1 

or pi > pi and w = e 
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Intended meaning: When the NLM M is in a situation K = (a, (yj ) , . . , (y t ) ) 6f / and nondeterministically 
chooses c 6 C for its current transition, then 

tape-config }>i (a(yi ) • • ■ (y t ) (c)) 

gives information on the inscription from tape cell /?! up to tape cell pi of the y'-th tape of the Turing 
machine's configuration at the end of M's current step. 

Induction base (i = 0): We start with M's start state orj and choose 

A := {ao}. 

If ao is final, then we let Kq :=<d and A := A(). This then gives us an NLM M which accepts its input without 
performing a single step. This is fine, since «o is nna l if> an d only if, the Turing machine T's start state qo is 
final, that is, T accepts its input without performing a single step. 
For the case that ao is not final, we let 

K : = | (a ,yi , . . ,y t ) | yi £ {(v) : v 6 /} and y 2 = •••=» = ()}■ 

The set Lq is defined via equation (Q. 

The function config is defined as follows: For every 

k = (ao,yi,--,y t ) € K {) 

with vi = (v) (for some v £ /), let 

t+u 

config Q (K) := (,„,Cl, v*©^^ 1 ), 

r-l 

Let 

{qo,Pi,--,P t ) ■= ao 

— ff T 11 

with pj = (pj,Pj,Pj,head-directionj), for all j G {1, . . ,f}. 

For j 6 {1, . . ,i} we define 



Now let c = (cj ,C2, • • ,ci) e C = be an arbitrary element from M's set C of nondeterministic choices. 
For defining U\ Ko (k,c) and tape-config j (a{yi) ■■■ (y t ){c)), consider the following: Let us start the Turing 
machine T with a configuration j\ that^Y.s to config a (K), i.e., that can be obtained from config Q (K) by replacing 
each occurrence of the wildcard symbol © by an arbitrary symbol in E. Let j\ , ji, 73, . . . be the successive 
configurations of T when started in J\ and using the nondeterministic choices 01,02,03,.. (in the sense of 
Definition [TTt. I.e., for all V > 1, y v +i is the (c v mod |Nextr(7v)|)-th of the |Nextr()V)| possible next 
configurations of y v . 

Using this notation, the definition of a^ Ka ( K", c) and 

tape-config jfi (a(yi) • • • (y t ) (c) ) 

can be taken verbatim from the definition of OCi^. +1 (k,c) and 

tape-config j M (a(yi) ■ ■ ■ (y,)(c)), 

given below. This completes the induction base (i = 0). 



Induction step ( i — * /+ 1 ): We let 

A i+ i ■- jfoeA 

and 



there are K G Ki and c £ C such that a^. (»C, c) = (b,e\, 
(for suitable (ei,..,e,) e Movement') 



a£A /+1 \S, 

yi 6 {(v) : ve/}UU?<;ii', and 

yje{{)}u\Ji'<iLf, for ail ye {2,.., 



The set L, + i is defined via equation Q. 

The function config i+l is defined as follows: Let c £ C and let K = (a,y\, . . ,y t ) 6 . Let 
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with pj = (pj, p\, p\, head-direction j), for all j € {1, . . ,f}, and 

4 = (q,Pt+i,--,Pt+u,w t +\,..,w, +u ). 

Let je {1,..J}. 

If yj £ L,' for some i' < i, then let 

(w>,p'J,p'J) := tape-configjjiyj). 

We choose := w'j. (This is well-defined, because tape-config j ,■/ and tape-config j/i operate identically on 
all elements in Lf nL,», for all i' , i" < i). 
Furthermore, we let be defined as follows: 

f(pj)P'J) ^ p\ = p\ = © and head-direction j = +1 
0' ! , P ; ) if P i =Pi = Q and head-direction ,■ = - 1 
IT 11 
(p],pj) otherwise. 

If y; ^ Ui'<;L,', then we make a case distinction on j: In case that j £ {2, . . ,f}, we have = {) and 
head- direction j = +1. We define as follows: 

M ■= M> 

and choose 

wj := ©^"W^-l) . 

In case that j = 1, we know that yj must be of the form (v), for some v £ /, and that head-direction j = +1. If 

v is not the m-th input item, that is, there is some . ,m— 1} such that (/i — 1) • (n+1) < p{ < jU • (n+1), 

then we define 

(pf,p|) := (pT )M .(n+l)), 

and choose 

Wx ■- ®(M-l)-("+l) v # ®<-M-(n+l) . 
Otherwise, v must be the m-th input item, that is, 

p} > (m-l)-(n+l). 

In this case we define 

(p\A) ■= (pU) 

and choose 

w , := ©(""-'Xn+l) v # n f-m-(n+l)_ 
If, for some jo£{l,..,l}, w j0 = e, then we define 

">nfig i+x {K) := ±, 
tape-configj,i+i(a{y\)---{yt){c)) := (e,2,l), 
and 0!|^. +1 (k", c) := (a, e'/, . .,ef), where for all j £ {1, . . ,f}, 

„ \ (head-direction j, true) if w/ = £ 

J | (head-direction j , false) otherwise. 

In what follows, we consider the case where Wj £, for all j £ {1, . . ,t}. We define 

config i+1 (K) := (q,pi,..,pt,Pt+i,--,Pt+u,wi,..,Wt,w t+ i,..,w t + u ), 

where g and p t +\,- ■ ,pt+ u ,w t+ \,. .,w t + u are obtained from q, 
pi,..,p t are obtained from a via p^ := pt, for all j £ {1, ..,<}, and 
wi,..,w t are chosen as above. 

Altogether, the description of the definition of config i+l (fc) is complete. 

For defining Otijf (f,c) and tape-config (a(y\) ■ ■ ■ (y t ){c)), consider the following: 
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Let us start the Turing machine T with a configuration j\ that fits to config i+l (fc), i.e., that can be obtained 
from config i+ i (k) by replacing each occurrence of the wildcard symbol ® by an arbitrary symbol in Z. Letting 

c = (ci,c2,..,q) e c = cf, 

we let 7i,72)73)--- be the successive configurations of T when started in yi and using the nondeterministic 
choices ci,C2,c$,.. (in the sense of DefinitionfTTIl. I.e., for all v > 1, Yv+l is the (c v mod |Next7-(y v )|)-fh of 
the |Next7(y v )| possible next configurations of y v . 

Then, there is a minimal v > 1 for which there exists a jo € {l,..,f,L} such that throughout the run 

YvYv-u 

(1) none of the heads l,..,t changes its direction, and 

(2) none of the heads j € {I, . . ,t} crosses a border pj or pj-, 
and one of the following cases applies: 

Case 1: jo ^ L, and in the transition from y v _i to y v , head jo crosses one of the borders p^ or pJ o . That is, 

in y v , the y'o _m head is either at position p^ — 1 or at position pJ Q + 1. 

(And none of the heads j G { 1 , . . , t} \ {jo } crosses a border or changes its direction^ ) 
Case 2: jo 7^ -L, an d m the transition from Yv-\ to y v , head y'o changes its direction, but does not cross one of 

the borders p'l or pi . 

'Jo 'Jo 

(And none of the heads j e { 1 , . . , ? } \ { 70 } crosses a border or changes its direction.) 
Case 3: y v is final and none of the cases 1 and 2 apply. Then we let jo := L. 
In all three cases we let 

1 11 11 11 11 11 \ 

We choose 



and define 



where 



will be specified below. 
Finally, we define 



where, for every j € {!,..,?}, 



f ■= (q",p" + i,--,p't'+ u y,' + i,--,w't'+u) 



b := ($",#,..,#), 
p'j = (p"],p"),p"^,head-direction" j 

a \K i+l (K,c) := {b,e'(,..,e' t '), 
e"j := (head-direction" , move'pj 



will be specified below. 

Recall that K" = (a, vi , . . , y,) 6 A", + i . For every j 6 {1, . . ,f} we define 
tape-confi gjJ+1 (a( yi }---(y t )(c}) := r «-J lj 



otherwise 



where pj and pj are specified below. 

For all j e {1, . . \ {70} we know (by the choice of v and 70) that throughout the Turing machine's compu- 
tation Y01 ■ ■ > Yv, head 7 neither changes its direction nor crosses one of the borders pj, pj. Consequently, we 



Recall that w.l.o.g. we assume that the Turing machine is normalized, cf. Definition|23l 
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choose 



head-direction": 



head-direction j 
false 



PJ 



P") 

P) 
„//T 



if head- direction j = + 1 
if head- direction j = — 1 

if head- direction j = + 1 
if head- direction j = — 1 

if head-direction ; = +1 



p '■ + 1 if head-direction j — — 1 



1 if head-direction j = +1 
if head- direction ; = — 1 



In Case 3 we have jo = _L, and therefore, ct^ (v,c) and tape-configj j+i(fl(yi) ••• (y?}(c}) is fully specified. 
Furthermore, note that in Case 3 we know that y v is final, i.e., </' is a final state of the Turing machine 7\ 
Therefore, b is a final state of the NLM M, and M's run accepts if, and only if, the simulated Turing machine 
run accepts (recall the definition of M's set of final and accepting states at the end of Step 3). 

For Case 1 and Case 2, we have jo € {1, . . ,t}, and for specifying 

head-direction'i , move":, p"\ , p"\ . p"\ , p\ , and p\ , 
JO' Jo ' 1 Jo ' ' Jo ' 1 Jo ' 1 Jo ■ 'Jo' 

we distinguish between the two cases: 



ad Case 1: In this case, jo ^= J_, and head 70 crosses one of the borders pj g or pj o in the transition from Yv-l 



to y v (that is, p" is either p\ + 1 or p [ \ — 1). We choose 



[[ „//T 



(P'% P' 



Jo' 1 Jo) 
move": 



head-direction'j o 



(©> P%' ©) 

(Pi Pi) 
true 



+1 1 fpX = pl + 1 

— 1 otherwise. 



ad Case 2: In this case, jo ^ _L, and head jo changes its direction, but does not cross one of the borders /3j o 

or pJ o . We only consider the case where the direction of head jo changes from +1 to — 1 (the other case is 

symmetric). 

We choose 



(head-direction"^ , move 



{p» 



Jo' 

Jo' P Jo' P Jo> 



(p\ 



{-I, false) 

(pIp'L'P'L- 



Note that here we might have pfj g + 1 = pj g . In this case, by the above definition, we obtain pj o = p'j o + 1 . 
Altogether, this completes the induction step. 

Finally, we are ready to fix M's state set A and transition function a as follows: 



A 
K 



;>o 

i>0 

U a \K, 

i>0 
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Note that 

1 . a is well-defined, because a\ K . and (X\g operate identical on all elements in [Ki n K? ) x C (for all i, i > 0). 

2. K consists of all situations {a,y\,..,y t ) € (A\B) x (A* )' that may occur in runs of M. 

3. a remains undefined for elements (a,y\, . . ,y t ) in (A\B) x (A*)' that do not belong to K. This is fine, 
because such a situation {a,y\, . . ,y t ) can never occur in an actual run of M. 

This completes Step 4. H 



Note that finally, the NLM M is fully specified. Due to the construction we know that M is (r,t) -bounded, 
because it has t lists and the number of head reversals during each run on an input v = (vj , . . , v m ) £ I m is 
bounded by the number r— 1 = r(m- (n+1)) — 1 of head reversals of the according run of the Turing machine 
T on input v\#- ■ ■ v m #. 



Step 5: For every input v = (vj , . . , v m ) £ I m we have 

Pr(M accepts VJ = Pr (T accepts \<\#- ■ -#v m ) . 

Proof: Let £^ £ N be an upper bound on the length of runs of the NLM M (such a number £m exists, because 
M is (r,f)-bounded; see Lemma[3T1tal( in Appendix IDl. 

For the remainder of this proof we fix an input v = (vj,. . ,v m ) € I" 1 for the NLM M and we let v := 
v\#- ■ ■ v m # denote the corresponding input for the Turing machine T. 

From Lemma[T8]we know that 

_ \{c T e C l T : p T {v,c T ) accepts}| _ \{c T eC e T : p T (v,c T ) accepts}| 
Pr(r accepts v) - ^ - ^ . 

Furthermore, we know from Lemma[25lthat 

_ . w \{c e C 1m : p M (v,c) accepts}| 
Pr(M accepts v) = Ji ^j^^ 

For showing that Pr(M accepts v) = Pr (T accepts v) it therefore suffices to show that 

\{c e C e " : Pm(v,c) accepts}| = |C| < ' M_1 • \{c T e C L T : p T {v,c T ) accepts} |. 
Consequently, it suffices to show that there is a function 

/ : C 1m -> dj- 

such that 

- for every c 6 C (m , the list machine run p^/(v,c) simulates the Turing machine run pj{v,f(c)), and 

- for every cj 6 Cy, 

|{ceC*» :/(c) = c r }| = ICI^- 1 . (8) 



We can define such a function / as follows: 
For every sequence 



c 



( 1 ),...,c^)) e c« 



following the construction of the NLM M in Steps 1-4, we obtain for each i 6 {1, . . , iy} that there is a uniquely 
defined prefix c^ of M's nondeterministic choice 



such that the following is true for 



(cf,..,cf) ec = c l T , 



5 m e m 



viewed as a sequence of elements from Cj- 

(1) The list machine run p^(v,c) simulates the Turing machine run pj(v,c), where M uses in its i-lh step 
exactly the cW -portion of cW for simulating the according Turing machine steps. 

(2) If t < £ denotes the length of the run pj(v,c) = (pi , • - ,P/), then c has exactly the length 

Now let i'o denote the maximum element from {1, . .,1m} such that |c('°'| ^ (in particular, this implies that 
£ = gW We let c('°> be the prefix of c('°> of length I- {I- 1 - |c('°'|) and define 

8 ■= ^■■■^B 1 ". 
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Note that, viewed as a sequence of elements from Cj, c has length exactly i, and therefore, we can well define 

m := s. 

Furthermore, to see that {8} is satisfied, note that / is surjective, i.e., for every c 6 C T there exists a c with 
f(c) = c, and 

\{ceC e » :/(c) =c}\ = \C T \" M - C = \C T \ HtM - l) = \C\ e "-\ 

(For the first equation, note that through c, exactly £ of the possible £ ■ £m Cj -components of c are fixed, 

whereas each of the remaining I-Im — £ components may carry an arbitrary element from Cj-) 

This completes Step 5. H 

Altogether, the proof of Lemma[76lis complete. □ 



D. DETAILED PROOF OF LEMMA 21 

This section is devoted to the proof of Lemma [2T1 

After pointing out an easy observation concerning randomized list machines in subsection lD.il we formally 
fix the notion of the skeleton of a list machine's run in subsection ID. 21 Then, in subsection ID. 31 we state 
and prove some basic properties of list machines concerning the size and shape of runs and the possibility of 
composing different runs. Afterwards, in subsection lD.4l we take a closer look at the information flow that can 
occur during a list machine's computation, and we show that only a small number of input positions can be 
compared during an NLM's run. Finally, in subsection |D3] we prove Lemmal2"Tl 

D.l An Easy Observation Concerning Randomized List Machines 

Lemma 26. Let M = (t,m,I,C,A,ao, a,B,B 

acc) b& cm NLM, let £ be an upper hound on the length of M s 
runs, and let Jf C I m such that Pr(M accepts v) > j, for all inputs v £ $ . Then there is a sequence c — 
[c\ , . . , C( ) 6 C such that the set 

Jaccx := {v<E / : Pm{v,c) accepts} 

has size \Jacc,c\ >\-\J\- 
Proof: By assumption we know that 

£ Pr(M accepts v) > 

From Lemma l25lwe obtain 

£ Pr(M accepts v) = £ 1 { ^ ^ P f { f 1 . 

Therefore, 

* \J\ 



£ \{c € C e : p M (v,c) accepts}! > \C e \ ■ li f 



On the other hand, 



Consequently, 



£ \{c £C f : Pm{v,c) accepts}| = £ |{v e f : p«(v,c) accepts} | . 



£ \{v e J : p M (v,c) accepts}! > |^|-^. 



cec 1 

Therefore, there must exist at least one ceC* with 



|{ve/:p M (v,c) accepts}! > ^ , 



and the proof of Lemma[26lis complete. 
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D.2 Skeletons of runs 



Definition 27 (local_views(p), ndet_choices(p), moves(p)). Let M be an NLM. 

(a) The local view, lv(y), of a configuration y = (a,p,d,X) of M is defined via 



My) : ~ ( a ,d,y) with y 



I.e., /v(y) carries the information on M's current state, head directions, and contents of the list cells cur- 
rently being seen, 
(b) Let p = (pi , . . , pi) be a run of M. We define 
(i) 

local_views(p) := (iv(pi), . . . ,/v(p^)). 

(ii) ndet_choices (p ) C C f ~ 1 to be the set of all sequences c = (ci , . . , q_ 1 ) such that, for all i < t, p, + 1 is 
the c,-successor of p,-. 

Note that Pr (p) = ^*HM. 

(iii) 

moves(p) := (movei, . . ,move£_i ) 6 ({0,1,-1}')' 

where, for every i < £, move,- = (move; i , . . ,move, jt ) 6 {0, 1,-1}' such that, for each t € {1,. . ,t}, 
move,. T = (resp., 1 , resp., — 1) if, and only if, in the transition from configuration p,- to configuration 
Pi+l, the head on the T-th list stayed on the same list cell (resp., moved to the next cell to the right, 
resp., to the left). H 

To prove lower bound results for list machines, we use the notion of a skeleton of a run. Basically, a skeleton 
describes the information flow during a run, in the sense that it does not describe the exchanged data items (i.e., 
input values), but instead, it describes which input positions the data items originally came from. The input 
positions of an NLM M = (t,m,I,C,A,ao, a,B,B acc ) are simply the indices i € {1, . . . -in}. 

Definition 28 (Index Strings and Skeletons). Let M be an NLM, let v = (\>\,..,v m ) e I m be an input for M, 
let p be a run of M for input v, and let y = (a,p,d,X) be one of the configurations in p. 

(a) For every cell content x t j in X (for each list T e {1, . •,«}), we write 

ind(x T j) 

to denote the index string, i.e., the string obtained from x T j by replacing each occurrence of input number 
v; by its index (i.e., input position) i € {1, . . ,m}, and by replacing each occurrence of a nondeterministic 
choice c € C by the wildcard symbol "?". 

(b) Fory= (xi yPl ,..,x, iPl ) T we let 

ind{y) := (ind(xi yPl ),..,ind(x t ,p,)) T . 

(c) The skeleton of a configuration fs local view lv(y) = (a,d,y) is defined via 

skel(lv(y)) := (a,d,ind(y)) . 

(d) The skeleton of a run p = (pi , . . , pi) of M is defined via 

skel(p) := (s,moves(p)) , 
where s = (si,. .,S() with s\ := skel(lv(p\)), and for all ( < I, if moves(p) = (movei,. . ,move£_i) T , 

skel(lv(p i+ i)) if move,- ^ (0,0, . . ,0) T 



otherwise. 



Remark 29. Note that, given an input instance v for an NLM M, the skeleton £ := skel(p) of a run p of M on 
input v, and a sequence c 6 ndet_choices(p), the entire run p can be reconstructed. H 

D.3 Basic Properties of List Machines 

In this section we provide some basic properties of list machines concerning the size and shape of runs, the 
number of skeletons of runs, and the possibility of composing different runs. 
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Lemma 30 (List length and cell size). 

Let M = (t,m,I,C,A,ao,OC,B,B acc ) bean (r, t) -bounded NLM. 

(a) The total list length of a configuration of M is defined as the sum of the lengths (i.e., number of cells) of 
all lists in that configuration^ 

For every i £ {1, . . ,r}, the total list length of each configuration that occurs before the i-th change of a 
head direction is < (t + 1)' ■ m. 

In particular, the total list length of each configuration in each run of M is < (t + l) r • m. 

(b) The cell size of a configuration of M is defined as the maximum length of the entries of the cells occurring 
in the configuration (remember that the cell entries are strings over A = /UCUAU {{,)})• 

The cell size of each configuration in each run of M is < 1 1 ■ (max{f, 2}) r . 

Proof: For deterministic list machines, (a) and (b) were proved in 1 10 1 (cf., Claims 1 and 2 in the proof of 1101 
Lemma 15]). For nondeterministic list machines, the proofs are virtually identical; only the cell size increases, 
as now the list entries also contain the nondeterministic choices. 

In fact, the proof of (a) is identical to the proof of 1101 Claim 2 in the proof of Lemma 15]: Let y be a 
configuration of total list length I. Then the total list length of a successor configuration y 1 of y is at most £ + t, 
if a head moves or changes its direction in the transition from y to y\ and it remains £ otherwise. 
Now suppose / is a configuration that can be reached from y without changing the direction of any head. Then 
/ is reached from y with at most I — t head movements, because a head can move into the same direction for 
at most A — 1 times on a list of length X . Thus the total list length of / is at most 

£ + t-(£-t). (9) 

The total list length of the initial configuration is m + 1 — 1 . A simple induction based on ^ shows that the 
total list length of a configuration that occurs before the i-th change of a head direction is at most 

O + iy'-m. 

This proves (a). 

For the proof of (b), let y be a configuration of cell size s. Then the cell size of all configurations that can be 
reached from y without changing the direction of any head is at most 

l+t-(2 + s) + 3 = 4 + t-(2 + s). 

The cell size of the initial configuration is 3. A simple induction shows that the total cell size of any configu- 
ration that occurs before the i-th change of a head direction is at most 

i-l 

4+^6^+5?' < 11- (max{?,2})''. 
7=1 

□ 

Lemma 31 (The shape of runs of an NLM). Let M = (t,m,I,C,A,ao,a,B,B acc ) bean (r,t)-bounded NLM, 
and let k := \A\. The following is true for every run p = (pi ... ,p(i) of M and the corresponding sequence 
moves (p) = (move i ,.. , movef_ i ). 

(a) I <k+k-(t + \) r+l -m. 

(b) There is a number )1 < (t + 1)' + I ■ m and there are indices 1 < j\ < ji < ■ ■ ■ < jn < I such that: 

(i) For every i 6 {1, .. ,£— 1}, 

move,/ (0,0,.., 0) T ^ / £ {ji , . . ,; M } . 

(ii) Ifn = Q, then t<k. 

Otherwise, j\ < k; j v + 1 — jv < k, for every v £ { 1 , . . , /x — 1 }; and £ — jfi < k. 

Proof: For indices i < £ with move; = (0,0, . . ,0) T we know from Definition [24]© that the state is the only 
thing in which p; and p, + i my differ. As (r, ?)-bounded NLMs are not allowed to have an infinite run, we obtain 
that without moving any of its heads, M can make at most k consecutive steps. 

On the other hand, for every ( e {1, . . , r} we know from Lemma[30ll[all that the total list length of a configu- 
ration that occurs before the i-th change of a head direction is 

< (t + iy-m. (10) 

Thus, between the (<— l)-st and the i-th change of a head direction, the number of steps in which at least one 
head moves is 

< (t + iy-m. 

4 Note that the total list length never decreases during a computation. 
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12-mit+l) lr+2 +24-(t+\y 



Altogether, for every run p of M, the total number of steps in which at least one head moves is 

r 

< £(f + l)*-m < (t+i) r+x -m. (11) 

Hence, we obtain that the total length of each run of M is 

< k+k-{t+\) r+l -m 

(namely, M can pass through at most k configurations before moving a head for the first time, it can move a 
head for at most (t + 1)' +1 • m times, and between any two head movements, it can pass through at most k 
configurations). 

Altogether, the proof of Lemma[3T|is complete. □ 
Lemma 32 (Number of Skeletons). 

Let M = (t,m,I,C,A,ao, a,B,B acc ) be an (r,t)-bounded NLM with t > 2 and k := \A\ > 2. 
The number 

\{skel(p) : p is a run ofM}\ 

of skeletons of runs of M is 

< (m + k + 3) 

Proof: We first count the number of skeletons of local views of configurations y of M. 
Let y be a configuration of M, and let /v(y) be of the form (a,d,y). Then, 

skel(lv{y)) = (a,d,ind(y)), 
where a S A, d S { — 1,1}', and ind(y) is a string over the alphabet 

{l,..,m}U{"?"}UAU{(,)}. 
Due to Lemma[30ll|bl, the string ind(y) has length < 11 ■ f. Therefore, 

\{skel(lv(j)) : yis a configuration of M}\ < k- 2' • (m + k + 3) lU ' . (12) 

From Lemma [3T1 we know that for every run p = (pi , . . ,p() of M there is a number fl < (t + l) r+1 • m and 
indices 1 < j\ < j% < ■ ■ ■ < jn < i such that for moves(p) = (movei , . . ,movef_i) we have: 

(i) For every is {1,..,^-1}, move/ ^ (0,0, .. ,0) T i S . 

(ii) K n = 0, then £ < k. 

Otherwise, j\ < k; j v+ i — j v < k, for every V S {1, . . , jU— 1}; and £ — jfi < k. 

The total number of possibilities of choosing such fx, I, j\ , . . , is 

(t+l) r+1 -m 

< £ k^+ l < k 2 +(' +i y +1 - m . (13) 

For each fixed p with parameters fl,£,ji,..,ju, skel(p) = (s, moves (p)) is of the following form: For every 
(' < £ with i ^ {ji,.. ,jfi}, move,- = (0,0, . . ,0) T and st+\ = "?". For the remaining indices j\ , . . there are 

< < 3 ('+iy +2 -m (14) 

possibilities of choosing (move^ , . . , move Jfi )S ({0, 1,-1}') M , and there arc 

< \{skel(lv(y)) : yis a configuration of M}\>* < (k ■ 2' ■ (m + k + 3) lU ') C +1 '' + ' ,m (15) 

possibilities of choosing (s jl + l , . . ,s jfl+i ) = (skel(lv(p jl + l )), . . ,skel(h{pj M + l ))). 

In total, by computing the product of the terms in | |13I >. l !14t , and J15t , we obtain that the number \ {skel(p) : 
p is a run of M} \ of skeletons of runs of M is at most 

(ja+fr+ir 1 *) . ( 3 ('+D' +2 -) . ((k-2i-(,n + k + 3) lUr ) i,+ir+ '- m ) 
n . f s2+(t+iy +2 -m 



< \k-3-k-2' ■ (m + k + 3) 



2 9/+log3 



k z -2 



■{m + k + 3)"-'') . (16) 

Obviously, 

k 2 < (k + m + 3) 2 . 
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< (m + k + 3) 

) (12.(/+l)').(2+(f+l)'-+ 2 . m ) 
i24-(f+l) r -|-12-(f+l) 2rW -m 



Since (k + m + 3) > 2 2 , we have 

2 '+l°g3 < ( m + k + 3)' +1 . 
Inserting this into l |16l l, we obtain that the number of skeletons of runs of M is 

I (Uf+t+3)-(2+(t+l) r+2 -m) 

< (m+k+3) 

< (m + k + 3) 

This completes the proof of Lemma[32] □ 
Definition 33. LetM= (t,m,I,C,A,ao,a,B,B acc ) be an NLM and let 

? = ((si,--,se), (move 1 ,..,move^_ 1 )) 

be the skeleton of a run p of M. We say that two input positions i, i' £ {1, . . ,m}, are compared in £ (respec- 
tively, in p) iff there is a 7 < I such that 5; is of the form 

skel(lv(y)) = (a,d,ind(y)), for some configuration y, 

and both i and i' occur in ind(y). H 

Lemma 34 (Composition Lemma). Let M = (t,m,I,C,A,aQ, a,B,B 

acc) be an NLM and let f GN be an upper 
bound on the length ofM's runs. Let £ be the skeleton of a run ofM, and let i, i' be input positions of M that 
are not compared in £. Letv = (v\,..,v m ) andw = (w\ , . . , W m ) be two different inputs for M with 

w j = v j> for all j e{l,..,m}\{i,i'} 

(i.e., V and w only differ at the input positions i and i'). Furthermore, suppose there exists a sequence c = 
(c\ , . . , q) EC' such that 

skel(p M (v,c)) = skel(p M (w,c)) = f, 

and Pm{v,c) and Pm(w,c) either both accept or both reject. Then, for the inputs u := (vi,. . ,v,-, . . ,Wf, . . ,v m ) 
and i( := (vi , . . , W;, . . , V? , . . , v m ) we have 

£ = skel(p M {u,c)) = skel(p M (Ti \c)) 

and 

Pm{u.c) accepts 4=> Pm("* ' ,c) accepts Pm(v,c) accepts <==> Pm(w,c) accepts. 

Proof: Let £ = ((si,...,sp ),(movei,...,move{i-\)) be the skeleton as in the hypothesis of the lemma. We 
show that skel(pM(u,c)) = £, and that Pm(u,c) accepts if and only if Pm(v,c) and Pm{w,c) accept. The proof 
for Ti' instead of u is the same. 

Let skel(pM(u, c)) = ((s\,...,Sf, l ),(move\,...,moveL,_ 1 )). Let j be the maximum index such that 

(i) (s' 1 ,...,s'j) = (si,...,sj), and 

(ii) (move'y, . . . ,move^_j) = (movei, . . . ,move 

Let / be the maximum index such that / < j and Sf= s'., 7^ "?". By the hypothesis of the lemma we know 
that i and t do not occur both in Sf, Thus for some x e {v,w}, s? contains only input positions where Ti 
and x coincide. Let p^/(x,c) = (pi,...,p#), and let Pm{u,c) = (p\,...,pL). Since Sf contains only input 
positions where u and x coincide, we have lv(pf) = lv(pj,). Since moveja = move'j,, = (0, ...,0) T for all 
/' £ {/, ... ,7 — 1}, we therefore have lv(pj) = Iv(p'j). This implies that the behavior in the y'-fh step of both 
runs, Pm{x,c) and Pm(u,c), is the same. 

Case 1 (j = £'): In this case there is no further step in the run, from which we conclude that K = I" . Hence 
both skeletons, £ and skel(pM(u,c)), are equal. Moreover, lv(pj) = lv(pj) implies that both runs either accept 
or reject. 

Case 2 (j < (' ): In this case we know that £" > j+ 1, and that movej = move'j. By the choice of jwe also have 
/+ 1 7^ s'j +l , which together with movej = move'j implies sj + \ 7^ "?" and 7^ "?". Let Sj+\ = (a,d,ind) 
and s'j +l = (a' ,d' ,ind'). Since lv(pj) = Iv(p'j), and the behavior in the j-th step of both runs is the same, we 
have a = a' and d = d! . So, ind and ind' must differ on some component l£ {l,...,f}. Let ind x be the T-th 
component of ind, and let ind x be the T-th component of ind . 

Since (s \ , . . . , Jj) = (s 1 , . . . , sj ) and (move'j , . . . , move'j ) = (move 1 , . . . , movej ) , the list cells visited directly 
after step / £ {0, . . . ,7} of all three runs, Pm{v,c), Pm(w,c) and Pm(u,c), are the same. This in particular 
implies that ind x and ind' x describe the same list cells, though in different runs. So, if ind x = (p) for some 
input position p, or ind x = {), then the cell described by ind x has not been visited during the first j steps of 



.v 
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all three runs, and therefore, ind' t = ind t . Now we may assume that ind t 7^ (p) for all input positions p, and 
indx 7^ {). Then, ind t = a(y\) . . . (y t )(c), where {a,d,y\,. . . ,y t ) = Sf for some /' £ {1, ... ,j}, and c is the 
/'-th nondeterministic choice of c. Also, ind' t = a' (y^) . . . (yj) (c), where (a',d',yj,.. . ,yj) = But sy» = 
which contradicts ind T 7^ ind' t . 

To conclude, only Case 1 can occur, which gives the desired result of the lemma. □ 

D.4 The information flow during a list machine's run 

In this subsection we take a closer look at the information flow that can occur during a list machine's compu- 
tation and, using this, we show that only a small number of input positions can be compared during an NLM's 
run. 

Definition 35 (subsequence). A sequence (si, .. ,s^) is a subsequence of a sequence (s\ ,..,s,,),if there exist 
indices ji < • • • < j A such that Sl = s^ , s 2 = s' h , ...,s^= . 

Definition 36. Let M = (t,m,I,C,A,ao,a,B,B acc ) be an NLM. Let y= (a,p,d,X) be a configuration of M 
withX = (xi , . . ,x t ) T and x T = (x x 1 , . . ,x T . nlt ), for each % £ {1, .. ,f}. Furthermore, let . .,f^) £ {1, . . ,m} , 
for some A £ N, be a sequence of input positions. 

We say that the sequence occurs in configuration y, if the following is true: There exists a 

T £ {l,..,f} and list positions 1 < j\ < • •• < 7'^ < m x such that, for all /i £ {1, . . , A}, the input position ip 
occurs in ind(x?j ). H 

The following lemma gives a closer understanding of the information flow that can occur during an NLM's 
run. 

Lemma 37 (Merge Lemma). Let M = (t,m,I,C,A,ao, a,B,B acc ) be an (r,t)-bounded NLM, let p be a run 
of M, let y be a configuration in p, and let, for some A £ N, £ {l,..,m} be a sequence of input 

positions that occurs in y. 

Then, there exist t r subsequences s\,..,Sp of {i\,. . ,ix) such that the following is true, where we let Sfi = 
{sii,\,--,s l i,X ll )J° r ever y P- £ { 1, • ■,''}■ 

f 

- {h,--,h} = U {■ ? Ma---^p.A ( ,}. and 

- for every G {1, . . ,t r }, s^ is a subsequence either of (1, . . ,m) or of (m, . . , 1). H 
Proof: By induction on r 1 6 {0, . . , r} we show that for each configuration that occurs during the r'-th scan 
(i.e., between the (/— l)-st and the r'-th change of a head direction), the above statement is true for f rather 
than f . 

For the induction start r 1 — we only have to consider M's start configuration. Obviously, every sequence 
(fl , . . , i^) that occurs in the start configuration, is a subsequence of (1, . . , m). 

For the induction step we note that all that M can do during the r'-th scan is merge entries from t different lists 
produced during the (/— l)-st scan. Therefore, {i\ , . . is the union of t sequences, each of which is a sub- 
sequence of either (j'i , . . , i^) or (i^, . . , i\) (corresponding to a forward scan or a backward scan, respectively), 
and each of these t subsequences has been produced during the (r 1 — l)-st scan. By induction hypothesis, each 
of these subsequences is the union of t r ~ 1 subsequences of (l,..,m) or (w,..,l). Consequently, (ii,..,/^) 
must be the union of t ■ f ~' such subsequences. □ 

We are now ready to show that only a small number of input positions can be compared during a list machine's 
run. 

Lemma 38 (Only few input positions can be compared by an NLM). 

Let M = (f , 2m,I,C,A,ao, a,B,B acc ) be an NLM with 2m input positions. 

Let v := (vi , . . , v m , Vj , . . , v^) G fi m be an input for M, let p be a run of M on input v, and let £ := skel(p). 
Then, for every permutation (p of { 1 , . . , m}, there are at most 

t 2r ■ sortedness((p) 

different i £ {1, . . ,m} such that the input positions i and m + (p(i) are compared in £ (i.e., the input values v, 
and v'mU) are compared in p ). 

Proof: For some A £ N let ;'i be distinct elements from {1, . . ,m} such that, for all fl £ {1,. . , A}, the 

input positions ip and m + (p(i^ ) are compared in £. From Definition 1331 and 1361 it then follows that, for an 
appropriate permutation % : { 1 , . . , A} — * {1, . . , A}, the sequence 

l := ( m+(p(i n(l) ) , i„(2), m+<p(i x{1) ) , ... , i n{ x) , m+(p(i K(k) ) ) 

occurs in some configuration in run p. From Lemma[37lwe then obtain that there exist f subsequences s\ ,. . ,s,r 
of I such that the following is true, where we let s^ = (s^i, • • ,s^ for every fl £ {1, . . ,t r }: 



25 



-{h,..,in,m + q>{h),..,m + <p(ix)} = [j {s^i , . . , K and 

- for every fie{l,.. ,f r }, ^ is a subsequence either of (1, . . ,2m) or of (2m, . . , 1). 

In particular, at least one of the sequences Si,. .,Sf must contain at least A' := [4] elements from {ij , . . ,i\}. 
W.l.o.g. we may assume that si is such a sequence, containing the elements 

Considering now the set {m-\-(p(i\) m+<p(i^i)}, we obtain by the same reasoning that one of the sequences 
s\ , . .,s t r must contain at least A" := [~4/| > elements from {m+<p(ii) , . . , m+<p(i^i)}. We may assume 
w.l.o.g. that S2 is such a sequence, containing the elements m+<p(h) , . . , m+<p(ix»)- 

Let us now arrange the elements i\ , . . , i^n ,m+(p(i\), . . ,m+<p(i\«) in the same order as they appear in the 
sequence l. I.e., let tc' : {1,. . , A"} — > {1, . . , A"} be a permutation such that 

t' : = ('jr'(l) » m +<P('V(l)) » ••• . ! V(A») i m +<P(^'(A")) ) 

is a subsequence of I . 

Since s\ is a subsequence of I and a subsequence of either (1, . . ,2m) or (2m, . . , 1), we obtain that 
either i n , (l) < ^ (2) < • • • < or ^(i) > %< (2 ) > ••• > *V(/L")- 

Similarly, since S2 is a subsequence of I and a subsequence of either (1, . . ,2m) or (2m, . . , 1), we obtain that 

either m+<p(v ( i)) < ••• < m+pfe,^,,)) or m+<p(v (1) ) > • • • > m+<p(v (A »)), 
and therefore, 

either <p(%'(l)) < '•• < <P(%'(A»)) or <P( ! V(l)) > ' ' ' > <P( ! V(A»))- 

In other words, (<p(V(i)), • • , <P('V(A"))) ls a subsequence of (<p(l), . . , <p(m)) that is sorted in either ascending 
or descending order. According to Definition[l9]we therefore have 

A" < sortedness((p) . 

Since A" > ^r, we hence obtain that 

A < t 2 ' ■ sortedness(<p) , 

and the proof of Lemma[38lis complete. □ 

D.5 Proof of Lemma EE 

Finally, we are ready for the proof of Lemmal2Tl 

Lemma l2l1(Lower Bound for List Machines) — restated. 

Let k,m,n,r,t £ N such that m is a power of 2 and 

t > 2, m > 24- (?+l) 4 '"+ 1, k > 2m + 3, n > 1 + (m 2 + 1) ■ log(2/fc). 

We let I := {0, 1}", identify I with the set {0, 1, . . ,2" — 1}, and divide it into m consecutive intervals I\,..,I m 
each of length 2" /m. 

Let (p be a permutation of {I, . . ,m\ with sortedness((p) < 2y / m— 1, and let 

■= ] <p(l) X Xl f(m) X h X ■■■ X/ m . 

Then there is no (r,t)-bounded NLM M = (t,2m,I,C,A,ao, a,B,B acc ) with \A\ < k and I = {0, 1}", such that 
for allv = (vi , . . , v m , v'j , . . , V m ) £ J* we have: 

Ifiyi, . . , v m ) = (v^/jj,. • , v^/ m j), fferc Pr(M accepts v) > \; otherwise Pr(M accepts v) = 0. Proof: Suppose 
for contradiction that M is a list machine which meets the requirements of Lemmal2T1 We let 

Seq ■= {{n,--,v m ,v[,..,v' m )eJ r ■. (n,--,v m ) = (v' (f , {l y,v' (p{m) )}. 

Note that 




From the lemma's assumption we know that 

Pr(M accepts v) > — , 

for every input v e J^ 9 . Our goal is to show that there is some input u 6 ^ \ ,i? eq , for which there exists an 
accepting run, i.e., for which Pr(M accepts u) > 0. It should be clear that once having shown this, the proof of 
Lemma[2"Tlis complete. 
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Since M is (r,t) -bounded, we know from Lemma[3"Tlthat there exists a number i £ N that is an upper bound on 
the length of M's runs. From Lemma[26lwe obtain a sequence c = (ci,..,c£) £ C 1 such that the set 

^accx ■= {ve/t, : Pm(v,c) accepts} 

has size 

1^1 1 _ • 



2 2 \ m 

Now choose £ to be the skeleton of a run of M such that the set 

^accx£ : = {vS^aecc ■ C = skel(p M (v,c)) } 

is as large as possible. 

I a _i J /2" x 



Claim 2. K c ^. f | > 



m 



(2/t)'"" - 2-(2fe)" 

Proof: Let tj denote the number of skeletons of runs of M. From Lemma[32lwe know that 

r, < (2m + ^ + 3) 24 "' (f+I)2 " +2+24 - ( ' +ir . 
From the assumption we know that k > 2m + 3, and therefore 

Tf < ( 2l t) 24 - m - ( ' +1)2r+2+24 - (f+1)r . (17) 
From the assumption ra > 24- (?+l) 4r + 1 we obtain that 

24-m-(? + l) 2 '' +2 +24-(/ + l)'' < 24-m-(f + l) 2r+2 + m < ra 2 . (18) 
Altogether, we obtain from d!7t and d!8t that 

r) < {2k) m2 . 

Since the particular skeleton £ was chosen in such a way that | J^ acc c r \ is as large as possible, and since the 
total number of skeletons is at most (2k)" r , we conclude that 

\J -A > ^ > —t—.(?T 

Hence, the proof of Claim[2]is complete. □ 

CLAIM 3. There is an j'q £ {1,-. , ra} skc/i //iaf f/ze rap;;/ positions ig a«rf m + <p(;'o) are not compared in £ . 

Proof: According to the particular choice of the permutation <p we know that 

sortedness((p) < 2-\fm— 1. 
Due to Lemma[38lit therefore suffices to show that m > t 2r ■ (2-y/m— 1). 

From the assumption that m > 24- (t + l) 4f + 1 we know that, in particular, m > 4 • ? 4r , i.e., .^/ra > 2 • t 2r . 
Hence, t 1 ' '■ (2-y/ra — 1) < j • y/m- (2y / m — 1) < ra, and the proof of Claim[3]is complete. □ 

Without loss of generality let us henceforth assume that !q = 1 (for other i'o, the proof is analogous but involves 
uglier notation). 

Now choose V2 £ 4,(2)' • • • » v m £ 4>(m) suc ^ mat 

| { v l e 4»(l) : ( v l, v 2--,v ffl ,v ? -i (1) ,v (p -i(2),--,v 9 -i( m )) £ J'acc^r } | 
is as large as possible. Then, the number of v\ such that 

(vi,V2..,V OT ,V v -i( l j,V y -i(2)i..,V 1 ,-l( m )) £ ^accxX 

is at least 

l^ecx.fl Cla ™H (f) 2" 



> 



(I)-' - 2.(2^.(1)'"-' - 2m.(2k)» 
From the assumption we know that n > 1 + (ra 2 + 1) • log( 2 Jfc). Therefore, 

2" > 2-{2k) ml+x > 2-(2k)-(2k) ml k >" 2-2m-(2k)" 
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Consequently, 

2" 

> 2. 

2m ■ (2k) m 

Thus, there are two different elements v\ 7^ W\ such that for (w2,..,w m ) := (v2,..,v m ) we have v := 
(vi,..,Vm ) v 9 -i(i),..,v 9 -i( JB )) € ^ acc ,c,j and W := (w\ , . . , w„„ w 9 - 1(1) , . . , w 9 -i (m) ) £ J^ CCi? ^. 
Since the run Pm(v,c) accepts, we obtain from Lemma[34lthat for the input 

u := (vi,..,v m ,w 9 -i( 1 ),..,w 9 -i( m )) G ^\^eq, 

the run Pm(", c) has to accept. Therefore, we have found an input Ti G \ =/e 9 with 

Pr(M accepts 17) > 0. 

This finally completes the proof of Lemma I2TI □ 

E. PROOFS OF LOWER BOUNDS FOR TURING MACHINES 

Proof of Corollary E) 

The upper bound is easily obtained when using a result of Chen and Yap 1 7 , Lemma 7] which states that the 
sorting problem (i.e., the problem of sorting a given sequence of strings) can be solved with two external 
memory tapes, O(logA') head reversals, and only constant internal memory space. 

The lower bound for the problems CHECK-SORT, Set-Equality, and MULTISET-EQUALITY is stated 
in Theorem [6] To obtain the according lower bound for the "SHORT" versions of these problems, we re- 
duce the problem CHECK-<p (cf., Lemmal22l to the problems SHORT-CHECK-SORT, SHORT-SET-EQUALITY, 
and SHORT-MULTISET-EQUALITY (that is, the restriction of these problems to inputs of the form Vi#- ■ • v m 
#v j #■■■ v' m #, where each v,- and each vj is a 0- 1 -string of length at most c ■ log m for some constant c > 2) in such 
a way that the reduction can be carried out in ST(0(l),0(logiV),2). More precisely, we construct a reduction 
(i.e., a function) / that maps every instance 

v := vi#- • • v m #v\#- ■ ■ v' m # 

of CHECK-<p to an instance 

M 

of Short-Check-Sort (respectively, of Short-Set-Equality or Short-Multiset-Equality), such 
that 

(1) the string /(v) is of length @(|v|), 

(2) /(v) is a "yes"-instance of Short-Check-Sort (respectively, a "yes"-instance of 
SHORT-(MULTI)SET-EQUALITY) if, and only if, v is a "yes"-instance of CHECK-<p, and 

(3) there is an (0(l),0(logA'),2)-bounded deterministic Turing machine that, when given an instance v of 
CHECK-<p, computes f(v). 

It should be clear that the existence of such a mapping / shows that if SHORT-CHECK-SORT (respectively, 
SHORT-(MULTI)SET-EQUALITY) belongs to the class RST(0(r),0(s),0(l)), for some s 6 ft(logiV), tri en 
also CHECK-ip belongs to RST(0(r),0(s),0(l)). If r and s are chosen according to the assumption of 
Corollary[7] this would cause a contradiction to Lemma|22] Therefore, SHORT-(MULTl)SET-EQUALITY and 

Short-Check-Sort do not belong to the class RST(o(logJV),0( 1 ^),0(l)). 
Now let us concentrate on the construction of the reduction /. 

For i € {1, .. ,m}, we subdivide the 0-1-string v,- 6 {0, l}" r into := [j^^] consecutive blocks v,- j , . . . , v,- 
each of which has length logm (to ensure that also the last sub block has length logm, we may pad it with lead- 
ing 0s). In the same way, we subdivide the string into sub blocks vj j , . . . , . For a number i£ { 1 , . . , m} 
we use BIN(z') to denote the binary representation of i—\ of length logm; and for a number j e we 
use BIN' (j) to denote the binary representation of j—\ of length 3 • log m. 
For every ( G {1, . . ,m} and j G {1, . . ,fl} we let 

w u := BIN(<p(0) BIN' (7) v Lj , 
BIN(i) BIN'(;) V Lj , 

W/,l#W i ',2#---vi ; !,M # ! 



w' 



hJ 

for every i G { 1 , . . , m} we let 



and finally, we define 

m 
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Clearly, f(y) can be viewed as an instance for Short-Check-Sort or SHORT-(MULTl)SET-EQUALITY, 
where m' := fl - m = fw^il pairs wy and w\ ■ of 0-1-strings of length 5 • logm < 2 • logm' are given. Let us 
now check that the function / has the properties (l)-(3). 

ad (1): Every instance v of CHECK-ip is a string of length N = ®{m - m 3 ) = 0(m 4 ), and f(v) is a string of 
length V = 0(m 4 ). 

ad (2): 

visa "yes"-instance of CHECK-<p 

(vi,..,v m ) = (v; (1) ,..,v' 9(m) ) 

(^-'(l)!--!^- 1 ^)) = ( v 'l''-' v m) 

forall/e{l,..,»i}, (vw.i-'-'VI'Im' = Ki'-'K/i)- ( 19 ) 

It is straightforward to see that fl9l > holds if, and only if, /(v) is a "yes"-instance of Short-(Multi)Set- 
EQUALITY. Furthermore, as the list of 0-1-strings in the second half of /(v) is sorted in ascending order, /(v) 
is a "yes"-instance of Short-Check-Sort if, and only if, it is a "yes"-instance of Short-(Multi)Set- 
EQUALITY. 

ad (3): In a first scan of the input tape, a deterministic Turing machine can compute the number m and store 
its binary representation on an internal memory tape. 

Now recall from Remark |20] that the permutation <p = <p m is chosen in such a way that for every i 6 {1, .. ,m}, 
the binary representation of <p(z) is exactly the reverse binary representation of i — and for each particular i, 
this can be computed on the internal memory tapes. Therefore, during a second scan of the input tape, the 
machine can produce the string /(v) on a second external memory tape (without performing any further head 
reversals on the external memory tapes). 

Altogether, the proof of Corollary|7]is complete. □ 
Proof of Corollary |9j 

(a) is an immediate consequence of Theorem[6]and Theorem [8] Ifaf. 

The second inequality in (b) follows directly from Theorem[6]and Theorem [8] ©. 

The first inequality in (b) holds because, due to Theorem [8] {a}, the complement of the MULTISET-EQUALITY 
problem belongs to RST(2, 0(logA'), 1). Since the deterministic ST(---) classes are closed under taking 
complements, Theorem [6] implies that the complement of the MULTISET-EQUALITY does not belong to 
ST(0(r),O(5),O(l)). □ 

Proof of Corollary HOl 

Of course, the CHECK-SORT problem can be solved for input xi#-- -x m #y\#- ■■y m # by (1) sorting x\#- ■ -#x m 
in ascending order and writing the sorted sequence, ■■#x' m onto the second external memory tape, and (2) 
comparing vj#- • -#y m and the (sorted) sequence x' l #- ■ ■#x' m in parallel. 

Therefore, if the sorting problem could be solved in LasVegas-RST(o(logN),0(-^^),0(l)), i.e., by an 

(o(logiV),0(j^),0(l))-bounded LasVegas-RTM T, then we could solve the CHECK-SORT problem by an 

(o(logiV),0( 1 ^),O(l))-bounded (^,0)-RTMr' which uses T as a subroutine such that T 1 rejects whenever 
T answers "/ don 't know " and T' accepts whenever T produces a sorted sequence that is equal to the sequence 
y i # ■ • ■ #y m . This, however, contradicts Theorem[6] □ 
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